Safe Transfer-Reinforcement-Learning-Based Optimal Control of Nonlinear Systems

Area of Science:

• Process control
• Artificial intelligence
• Chemical engineering

Background:

• Traditional reinforcement learning (RL) for optimal control of nonlinear processes suffers from high computational demands and safety concerns during training.
• Ensuring closed-loop system safety is critical but challenging with existing RL methods.

Purpose of the Study:

• To propose a safe transfer reinforcement learning (TRL) framework to accelerate learning and improve safety in optimal control of nonlinear processes.
• To reduce computational resource requirements and training durations for control policy optimization.

Main Methods:

• Developed a TRL algorithm that utilizes knowledge from pretrained source tasks for faster learning on new target tasks.
• Implemented a control invariant set (CIS) to guarantee safety during data collection and policy optimization.
• Provided theoretical analysis of policy errors, considering source and target task discrepancies.

Main Results:

• The TRL framework significantly reduces learning time and computational resources compared to traditional RL.
• Safety of the closed-loop system is guaranteed throughout the learning process via the CIS.
• Demonstrated effectiveness in chemical process case studies, achieving efficient optimal control with guaranteed safety.

Conclusions:

• The proposed safe TRL framework offers an effective solution for computationally efficient and safe optimal control of nonlinear processes.
• Leveraging prior knowledge and maintaining safety within a CIS are key to overcoming limitations of traditional RL.
• The method shows promise for real-world applications, particularly in complex systems like chemical processes.