Adaptive control for circulating cooling water system using deep reinforcement learning

Area of Science:

• Control Engineering
• Artificial Intelligence
• Water Systems Management

Background:

• Circulating cooling water systems present complex control challenges.
• Traditional control methods often fail to achieve stable and precise system regulation.
• The multivariable nature of these systems necessitates advanced control strategies.

Purpose of the Study:

• To introduce a novel adaptive control structure, TD3-RTM, for improved circulating cooling water system control.
• To leverage deep reinforcement learning for enhanced system stability and precision.
• To reduce oscillations and instability in the control system using a reference trajectory model.

Main Methods:

• Developed a deep reinforcement learning agent using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm.
• Incorporated a reference trajectory model (RTM) into the TD3 algorithm, creating TD3-RTM.
• Defined a state space and designed a dense reward function tailored to the system's multivariable characteristics.
• Conducted simulation experiments in MATLAB/Simulink to evaluate performance.

Main Results:

• TD3-RTM significantly improved transient time in both flow and temperature loops compared to PID, fuzzy PID, DDPG, and TD3.
• The Integral of Absolute Error (IAE) was substantially reduced across all tested loops.
• Overshooting in the flow loop was notably decreased compared to PID, fuzzy PID, and TD3.

Conclusions:

• The TD3-RTM method offers superior performance in stabilizing circulating cooling water systems.
• This adaptive control structure effectively addresses the limitations of traditional control methods.
• TD3-RTM demonstrates potential for optimizing the operation of complex industrial water systems.