IEEE Transactions on Smart Grid · 2021 · 149 citations · 34 references
ComputingIndustrial EngineeringDistributed SystemsMachine LearningOptimizationDistributed ControlDistributed Energy ResourcesPower ControlReinforcement LearningSmart Grid SystemsSmart GridsActive Distribution NetworksConstrained Multi-agent Reinforcement LearningConstrained Reinforcement LearningControl Framework DesignEnergy Distribution NetworksMulti-agent LearningMulti-agent Reinforcement LearningReal-time Communication
The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs), which is based on perfect model and real-time P2P communication. However, the model is always incomplete with significant parameter errors and such P2P communication system is hard to maintain. In this article, we propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC. In this framework, the VVC problem is formulated as a constrained Markov game and we propose a novel multi-agent constrained soft actor-critic (MACSAC) reinforcement learning algorithm. MACSAC is used to train the control agents online, so the accurate ADN model is no longer needed. Then, the trained agents can realize decentralized optimization using local measurements without real-time P2P communication. The OLDC with MACSAC has shown extraordinary flexibility, efficiency and robustness to various computing and communication conditions. Numerical simulations on IEEE test cases not only demonstrate that the proposed MACSAC outperforms the state-of-art learning algorithms, but also support the superiority of our OLDC framework in the online application.
34
Soft Actor-Critic Algorithms and Applications
Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen et al. · arXiv (Cornell University) · 2018 · 2K citations · Full text