Combating Uncertainties in Smart Grid Decision Networks: Multiagent Reinforcement Learning With Imperfect State Information

Abstract

Renewable energy sources, such as wind and solar power, are increasingly being integrated into smart grid systems. However, when compared to traditional energy resources, the unpredictability of renewable energy generation poses significant challenges for both electricity providers and utility companies. Furthermore, the large-scale integration of distributed energy resources (such as photovoltaics (PV) systems) creates new challenges for energy management in microgrids. To tackle these issues, we consider a framework with two objectives: (i) combating uncertainty of renewable energy in smart grid by leveraging time-series forecasting with Long-Short Term Memory (LSTM) solutions, and (ii) establishing distributed and dynamic decision-making framework with multi-agent reinforcement learning (RL) with uncertain and imperfect state information. The proposed framework addresses these objectives while considering both wholesale and retail markets, thereby enabling efficient energy management in the presence of uncertain and distributed renewable energy sources. Through extensive numerical simulations based on the Deep Deterministic Policy Gradient (DDPG) RL algorithm, we demonstrate that the proposed solution significantly improves the profit of load serving entities (LSE) by providing a more accurate wind generation forecast. Furthermore, our results demonstrate that the households with PV and battery installations can increase their profits by using intelligent battery charge/discharge actions determined by the DDPG agents.