Adaptive Safe Reinforcement Learning With Full-State Constraints and Constrained Adaptation for Autonomous Vehicles

Area of Science:

• Autonomous Systems
• Machine Learning
• Control Theory

Background:

• Safety-critical autonomous vehicles require state variables to remain within defined regions during learning.
• Existing reinforcement learning (RL) methods may not guarantee safety constraints throughout the learning process.

Purpose of the Study:

• To develop an adaptive safe reinforcement learning (RL) algorithm for autonomous vehicles.
• To ensure full-state variables are constrained within the safety region during the entire learning process.
• To enhance control performance and safety assurance in autonomous vehicle systems.

Main Methods:

• An adaptive safe RL algorithm integrating an optimized backstepping technique and asymmetric barrier Lyapunov function (BLF) methodology.
• Decomposition of subsystem control and value function derivatives with BLF-related terms and independent learning components.
• A constrained adaptation algorithm with a projection operator to manage safety-optimization conflicts.

Main Results:

• The proposed algorithm ensures full-state variables remain within the safety region during learning.
• Demonstrated superior performance in simulations for autonomous vehicle motion control compared to existing methods.
• Verified improved convergence and reduced variance, particularly under uncertain conditions.

Conclusions:

• The adaptive safe RL algorithm effectively optimizes system control while guaranteeing state variable constraints.
• The methodology provides doubly assured safety performance through constrained adaptation.
• The approach is validated for enhancing the safety and performance of autonomous vehicle control systems.