Multi-Agent Deep Reinforcement Learning Method for EV Charging Station Game

Abstract

The ongoing quest for transportation electrification with the massive proliferation of EV charging stations (EVCSs) will deepen the interaction and require the further coordination of coupled power and transportation networks (PTN). The individually-owned EVCSs located in an urban transportation network (UTN) will compete using price signals to maximize their respective payoffs. In this paper, a multi-agent deep reinforcement learning (MA-DRL) method is proposed to model the pricing game in UTN and determine the optimal charging prices for a single EVCS. The EVCS charging demand is first analyzed using a modified user equilibrium traffic assignment problem (UE-TAP) with elastic traveling demands and different charging prices. The price competition problem is then formulated as a game with incomplete information in which the market environment is complex due to nonlinear traffic assignments. Thus, the MA-DRL approach is proposed to learn the charging pricing strategies of multiple EVCSs and approximate the Nash Equilibrium (NE) of the pricing game using the incomplete information. The proposed solution will determine the optimal pricing strategies for an EVCS in UTN. The case studies on a 24-node Sioux-Falls network, and the real-world Xi'an and Hangzhou cities are conducted to verify the effectiveness and scalability of the proposed approach.