Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control.

Area of Science:

• Process Control
• Artificial Intelligence
• Intelligent Manufacturing

Background:

• Traditional model-based control struggles with changing industrial conditions.
• Reinforcement learning (RL) offers model-free control but lacks transfer learning for mode variations.
• Existing RL algorithms face challenges in adapting to dynamic industrial environments.

Purpose of the Study:

• To develop an RL framework that improves training efficiency and transfer learning for industrial process control.
• To address the limitations of current RL methods in handling drastic changes in operating modes.
• To propose a novel RL algorithm capable of rapid adaptation to environmental shifts.

Main Methods:

• Designed a framework using local data augmentation for enhanced training and transfer learning.
• Developed the Case-Based Reasoning Model-Assisted Deep Deterministic Policy Gradient (CBR-MA-DDPG) algorithm.
• Integrated case-based reasoning (CBR) and model-assisted (MA) experience augmentation with DDPG.

Main Results:

• CBR-MA-DDPG demonstrated rapid adaptation to varying environments within a few training episodes.
• Experimental validation on CSTR and ORC systems showed superior adaptability and control performance.
• Outperformed conventional PI and MPC control schemes in performance and robustness.

Conclusions:

• The proposed CBR-MA-DDPG algorithm significantly improves transfer learning efficiency and adaptability in industrial process control.
• Achieves superior control performance and robustness compared to existing RL and traditional methods.
• Offers a viable solution for intelligent manufacturing under dynamic operating conditions.