Deep Reinforcement Learning-Based Hierarchical Energy Control Strategy of a Platoon of Connected Hybrid Electric Vehicles Through Cloud Platform

Abstract

Due to the features of strong nonlinearity and hybrid driving with multipower sources, a novel deep reinforcement learning (DRL)-based hierarchical energy control architecture is presented for a group of connected hybrid electric vehicles (HEVs) in the platoon through cloud platform. First, a higher-level model predictive control (MPC) law is designed to determine the desired acceleration of connected HEVs in a platoon. Second, a reward function with the change of state of charge (SOC) and instantaneous fuel consumption as independent variables is constructed, and the expert knowledge-based deep deterministic policy gradient (DDPG) method with prioritized experience replay (PER) is used for designing the lower-level energy management strategy of connected HEVs platooning. Then, to illustrate the rationality of decision-making by the presented DDPG strategy, the influence of state variables such as vehicle speed, desired acceleration, and battery SOC on the agent action values of the presented DDPG method is discussed. Finally, the test results show that the proposed control scheme can reasonably allocate the engine and motor powers in real time, and finally achieve the safe and energy-saving driving of connected HEVs in the platoon.