Reinforcement Learning in Pursuit-Evasion Differential Game: Safety, Stability, and Robustness

Area of Science:

• Robotics and Control Systems
• Artificial Intelligence
• Optimization Theory

Background:

• Safety and stability are paramount in pursuit-evasion (PE) problems within complex environments.
• Existing reinforcement learning (RL) and control barrier function (CBF) approaches often neglect environmental disturbances like wind gusts or actuator faults.
• Addressing these disturbances is crucial for practical applications.

Purpose of the Study:

• To develop a safe and robust RL framework for PE problems.
• To integrate CBFs and sliding mode control (SMC) within an RL paradigm to ensure safety, stability, and robustness against disturbances.
• To overcome the coupling challenges between CBF and SMC terms.

Main Methods:

• A hierarchical design scheme inspired by Stackelberg games is proposed, treating CBF as the leader and SMC as the follower.
• The PE problem is formulated as a zero-sum game.
• A safe, robust RL framework is developed to learn a min-max strategy online.
• A sufficient condition for algorithm effectiveness under conflicting constraints is provided.

Main Results:

• The proposed hierarchical approach effectively manages the coupling between CBF and SMC.
• The framework demonstrates robustness against external disturbances.
• Simulation results validate the effectiveness of the safe, robust RL approach.
• The algorithm maintains effectiveness even with conflicting constraints.

Conclusions:

• The developed safe, robust RL framework successfully addresses safety, stability, and robustness in PE problems with disturbances.
• The hierarchical Stackelberg game-inspired design is key to managing complex control interactions.
• The approach offers a promising solution for real-world applications requiring reliable autonomous systems.