Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

TLDR

In commercial buildings, HVAC systems consume 40–50 % of electricity, yet unknown thermal dynamics, parameter uncertainties, coupled constraints, and a non‑convex objective make energy‑cost minimization highly challenging. The study aims to minimize HVAC energy cost in a multi‑zone commercial building while accounting for random occupancy, thermal comfort, and indoor air quality. The authors reformulate the problem as a Markov game and solve it with a multi‑agent deep reinforcement learning algorithm that incorporates an attention mechanism and requires no prior knowledge of building dynamics. Simulations using real‑world traces demonstrate that the algorithm is effective, robust, and scalable.

Abstract

In commercial buildings, about 40%-50% of the total electricity consumption is attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems, which places an economic burden on building operators. In this paper, we intend to minimize the energy cost of an HVAC system in a multi-zone commercial building with the consideration of random zone occupancy, thermal comfort, and indoor air quality comfort. Due to the existence of unknown thermal dynamics models, parameter uncertainties (e.g., outdoor temperature, electricity price, and number of occupants), spatially and temporally coupled constraints associated with indoor temperature and CO2 concentration, a large discrete solution space, and a non-convex and non-separable objective function, it is very challenging to achieve the above aim. To this end, the above energy cost minimization problem is reformulated as a Markov game. Then, an HVAC control algorithm is proposed to solve the Markov game based on multi-agent deep reinforcement learning with attention mechanism. The proposed algorithm does not require any prior knowledge of uncertain parameters and can operate without knowing building thermal dynamics models. Simulation results based on real-world traces show the effectiveness, robustness and scalability of the proposed algorithm.