Residual Physics and Post-Posed Shielding for Safe Deep Reinforcement Learning Method.

Area of Science:

• Computer Science
• Artificial Intelligence
• Thermodynamics

Background:

• Deep reinforcement learning (DRL) shows promise for data center (DC) air conditioning control.
• Current DRL applications face challenges with data requirements and ensuring safe operation within temperature constraints.

Purpose of the Study:

• To propose a novel control method, RP-SDRL, addressing DRL limitations in DC cooling.
• To enhance DRL's sample efficiency, robustness, and safety in critical systems.

Main Methods:

• Integrated Residual Physics (based on thermodynamics) with DRL and a Prediction Model.
• Employed a gradient descent-based Correction Model for Post-Posed Shielding to ensure safe actions.
• Validated the RP-SDRL method through simulations with added state uncertainty (noise).

Main Results:

• RP-SDRL significantly improved DRL's initial policy, sample efficiency, and robustness.
• Residual Physics enhanced sample efficiency and prediction model accuracy.
• RP-SDRL effectively detected unsafe actions and reduced constraint violations compared to DRL alone.
• Achieved approximately 13% electricity savings compared to a baseline controller.

Conclusions:

• The proposed RP-SDRL method offers a viable solution for safe and efficient DC cooling control.
• Combining physics-based principles with DRL enhances performance and reliability in critical infrastructure.
• RP-SDRL demonstrates potential for substantial energy savings in data center operations.