Reinforcement learning for whole-building HVAC control and demand response

TLDR

Prior work has applied RL to HVAC control in isolation, relying on simplified models that focus on energy efficiency while largely ignoring demand response. This study introduces a holistic RL framework that simultaneously optimizes HVAC scheduling for energy efficiency, thermal comfort, and demand response. The approach employs a smart autonomous RL agent that adjusts HVAC set points in real time to reshape building electricity usage in response to demand response signals while preserving occupant comfort. Simulations demonstrate that the RL controller can cut weekly energy use by up to 22% versus a handcrafted baseline, and during demand response events can achieve up to 50% power savings or increases relative to a standard RL controller while maintaining comfort.

Abstract

This paper proposes a novel reinforcement learning (RL) architecture for the efficient scheduling and control of the heating, ventilation and air conditioning (HVAC) system in a commercial building while harnessing its demand response (DR) potentials. With advances in automated building management systems, this can be achieved seamlessly by a smart autonomous RL agent which takes the best action, for example, a change in HVAC temperature set point, necessary to change the electricity usage pattern of a building in response to demand response signals, and with minimal thermal comfort impact to customers. Previous research in this area has tackled only individual aspects of the problem using RL. Specifically, due to the challenges in implementing demand response with whole-building models, simpler analytical models which poorly capture reality have been used instead. And where whole-building models are applied, RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected. Thus, in this research, we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals. Our simulation results show that by applying reinforcement learning for normal HVAC operation, a maximum weekly energy reduction of up to 22% can be achieved compared to a handcrafted baseline controller. Furthermore, by employing a DR-aware RL controller during demand response periods, average power reductions or increases of up to 50% can be achieved on a weekly basis compared to the default RL controller, while keeping occupant thermal comfort levels within acceptable bounds.