Deep reinforcement learning for energy management in a microgrid with flexible demand

TLDR

The study evaluates deep reinforcement learning algorithms for optimizing energy management in a microgrid. The authors model a microgrid comprising wind generation, storage, thermostatically controlled and price‑responsive loads, and a grid link, and implement seven deep reinforcement learning algorithms to coordinate these resources. Results show wide variation in convergence among algorithms, with an enhanced A3C variant—using experience replay and semi‑deterministic training—achieving the best performance and near‑optimal policies.

Abstract

In this paper, we study the performance of various deep reinforcement learning algorithms to enhance the energy management system of a microgrid. We propose a novel microgrid model that consists of a wind turbine generator, an energy storage system, a set of thermostatically controlled loads, a set of price-responsive loads, and a connection to the main grid. The proposed energy management system is designed to coordinate among the different flexible sources by defining the priority resources, direct demand control signals, and electricity prices. Seven deep reinforcement learning algorithms were implemented and are empirically compared in this paper. The numerical results show that the deep reinforcement learning algorithms differ widely in their ability to converge to optimal policies. By adding an experience replay and a semi-deterministic training phase to the well-known asynchronous advantage actor–critic​ algorithm, we achieved the highest model performance as well as convergence to near-optimal policies.