A Deep Reinforcement Learning Method for Pricing Electric Vehicles With Discrete Charging Levels

Abstract

The effective pricing of electric vehicle (EV) charging by aggregators constitutes a key problem toward the realization of the significant EV flexibility potential in deregulated electricity systems and has been addressed by previous work through bi-level optimization formulations. However, the solution approach adopted in previous work cannot capture the discrete nature of the EV charging/discharging levels. Although reinforcement learning (RL) can tackle this challenge, state-of-the-art RL methods require discretization of state and/or action spaces and thus exhibit limitations in terms of solution optimality and computational requirements. This article proposes a novel deep reinforcement learning (DRL) method to solve the examined EV pricing problem, combining deep deterministic policy gradient (DDPG) principles with a prioritized experience replay (PER) strategy and setting up the problem in multi-dimensional continuous state and action spaces. Case studies demonstrate that the proposed method outperforms state-of-the-art RL methods in terms of both solution optimality and computational requirements and comprehensively analyze the economic impacts of smart-charging and vehicle-to-grid (V2G) flexibility on both aggregators and EV owners.