Control-Informed Reinforcement Learning for Chemical Processes.

Area of Science:

• Control Theory
• Machine Learning
• Artificial Intelligence

Background:

• Classical control methods like PID excel at disturbance rejection and setpoint tracking.
• Deep reinforcement learning (RL) offers powerful nonlinear modeling capabilities.
• Integrating these approaches can overcome individual limitations.

Purpose of the Study:

• To propose a novel control-informed reinforcement learning (CIRL) framework.
• To leverage prior knowledge from control theory within deep RL policies.
• To enhance the performance, robustness, and generalization of RL agents.

Main Methods:

• Developed a CIRL framework integrating Proportional-Integral-Derivative (PID) control components into deep RL policy architecture.
• Incorporated prior knowledge from control theory into the learning process.
• Validated the framework using simulation studies on a continuously stirred tank reactor system.

Main Results:

• CIRL demonstrated superior performance compared to conventional model-free deep RL and static PID controllers.
• CIRL exhibited enhanced set-point tracking, especially for out-of-distribution trajectories, indicating improved generalization.
• The embedded control knowledge improved robustness against unobserved system disturbances.

Conclusions:

• The CIRL framework effectively combines strengths of classical control and deep RL.
• CIRL offers a sample-efficient and robust approach for complex industrial systems.
• This hybrid approach shows significant potential for advanced control applications.