Energy and AI · 2022 · 28 citations · 31 references
Natural ResourcesAutonomous SystemsEconomicsEnergy EconomicsEnergy ManagementEnergy SystemsDemand ResponseDistributed Energy ResourcesElectricity MarketsEnergy PricingEnergy StorageReinforcement LearningDouble Auction MarketsEnergy Market DesignLearning AgentsLearning In Multi-agent SystemsMarket Clearing MechanismsMarket SimulationMulti-agent LearningMulti-agent Reinforcement Learning
Local energy markets are emerging as a tool for coordinating generation, storage, and consumption of energy from distributed resources. In combination with automation, they promise to provide an effective energy management framework that is fair and brings system-level savings. The cooperative–competitive nature of energy markets calls for multi-agent based automation with learning energy trading agents. However, depending on the dynamics of the agent–environment interaction, this approach may yield unintended behavior of market participants. Thus, the design of market mechanisms suitable for reinforcement learning agents must take into account this interplay. This article introduces autonomous local energy exchange (ALEX) as an experimental framework that combines multi-agent learning and double auction mechanism. Participants determine their internal price signals and make energy management decisions through market interactions, rather than relying on predetermined external price signals. The main contribution of this article is examination of compatibility between specific market elements and independent learning agents. Effects of different market properties are evaluated through simulation experiments, and the results are used for determine a suitable market design. The results show that market truthfulness maintains demand-response functionality, while weak budget balancing provides a strong reinforcement signal for the learning agents. The resulting agent behavior is compared with two baselines: net billing and time-of-use rates. The ALEX-based pricing is more responsive to fluctuations in the community net load compared to the time-of-use. The more accurate accounting of renewable energy usage reduced bills by a median 38.8% compared to net billing, confirming the ability to better facilitate demand response.
31
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Oriol Vinyals, I. Babuschkin, Wojciech Marian Czarnecki et al. · Nature · 2019 · 3.4K citations
Efficient mechanisms for bilateral trading
Roger B. Myerson, Mark A. Satterthwaite · Journal of Economic Theory · 1983 · 2.6K citations
Tom Schaul, John Quan, Ioannis Antonoglou et al. · arXiv (Cornell University) · 2015 · 2K citations · Full text