Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning

Abstract

This paper explores the use of distributed intelligence to assist the integration of the demand as a flexible resource, to mitigate the emerging uncertainty in the power system, while fulfilling the customer's local needs, i.e., comfort management. More exactly, our contribution is twofold. First, we propose a novel cooperative and decentralized reinforcement learning method, dubbed extended joint action learning (eJAL). Second, we perform a comparison between eJAL to noncooperative decentralized decision making strategies, i.e., Q-learning, and a centralized game theoretic approach, i.e., Nash n-player game. This comparison has been conducted on the basis of grid support effectiveness and the loss of comfort for each customer. Various metrics were used to analyze the advantages and disadvantages of each method. We demonstrated that a range of flexibility requests can be met by providing an optimal energy portfolio of buildings without substantially violating comfort constraints. Moreover, we showed that the proposed eJAL method achieves the highest fairness index.