Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for safe reinforcement learning in...

Area of Science:

• Control Engineering
• Machine Learning
• Artificial Intelligence

Background:

• Optimal control of nonlinear systems presents challenges in safety and scalability.
• Reinforcement learning (RL) often struggles with efficient memory management and maintaining diverse experiences.

Purpose of the Study:

• To propose a novel RL framework, SODACER, for safe and scalable optimal control of nonlinear systems.
• To integrate experience replay mechanisms with safety constraints for robust learning.

Main Methods:

• Developed Self-Organizing Dual-buffer Adaptive Clustering Experience Replay (SODACER) with Fast and Slow Buffers.
• Implemented a self-organizing adaptive clustering mechanism for experience replay memory optimization.
• Integrated SODACER with Control Barrier Functions (CBFs) for safety guarantees and the Sophia optimizer for enhanced convergence.

Main Results:

• SODACER demonstrated faster convergence and improved sample efficiency compared to baseline methods.
• The framework achieved a superior bias-variance trade-off while ensuring safe system trajectories.
• Validated effectiveness on a nonlinear Human Papillomavirus (HPV) transmission model with safety constraints.

Conclusions:

• SODACER provides a reliable, effective, and robust learning architecture for dynamic, safety-critical environments.
• The approach offers a generalizable solution for robotics, healthcare, and large-scale system optimization.
• The proposed method significantly enhances learning performance and safety in complex control tasks.