Efficient and assured reinforcement learning-based building HVAC control with heterogeneous expert-guided training.

Area of Science:

• Building energy systems
• Artificial intelligence in control engineering
• Sustainable building technologies

Background:

• Building HVAC systems consume significant energy (nearly 50% in the US) and impact occupant well-being.
• Model-free deep reinforcement learning (DRL) offers an alternative to traditional HVAC control but faces challenges with long training times.

Purpose of the Study:

• To develop a systematic approach for accelerating online reinforcement learning for HVAC control.
• To leverage domain expertise to overcome the training time limitations of DRL in HVAC applications.

Main Methods:

• Learning expert functions from abstract physical models and historical data using offline reinforcement learning.
• Integrating expert functions with rule-based guidelines for guided training and policy initialization.
• Implementing a runtime shielding framework to minimize temperature violations.

Main Results:

• Achieved up to an 8.8X speedup in DRL training compared to previous methods.
• Demonstrated a low temperature violation rate, ensuring occupant comfort.
• Successfully integrated expert knowledge into the DRL training process.

Conclusions:

• The proposed systematic approach effectively accelerates DRL training for HVAC control.
• This method enhances the practical deployment of DRL in energy-efficient building management.
• Expert knowledge integration is key to improving DRL efficiency and performance in HVAC systems.