Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning.

Area of Science:

• Artificial Intelligence
• Robotics
• Autonomous Systems

Background:

• Reinforcement Learning (RL) is effective for autonomous driving but struggles with robust safety in diverse data, especially long-tail scenarios.
• Ensuring safety and considering data distribution variations are critical challenges for RL-based decision-making in autonomous vehicles.

Purpose of the Study:

• To present a novel framework for highway autonomous driving that prioritizes both safety and robustness.
• To develop a method that updates RL strategies to maximize rewards while adhering to safety constraints.

Main Methods:

• Introduced Replay Buffer Constrained Policy Optimization (RECPO) to update RL strategies within safety constraints.
• Utilized importance sampling and a Replay buffer for data reutilization, mitigating catastrophic forgetting.
• Formulated the problem as a Constrained Markov Decision Process (CMDP) for policy optimization.

Main Results:

• The RECPO framework demonstrated significantly enhanced model convergence speed, safety, and decision-making stability in CARLA simulations.
• Achieved a zero-collision rate in highway autonomous driving scenarios.
• Outperformed traditional CPO, Deep Deterministic Policy Gradient (DDPG), and Intelligent Driver Model + MOBIL (IDM + MOBIL) methods.

Conclusions:

• The proposed RECPO method offers a robust and safe approach for training autonomous driving policies.
• The framework effectively addresses safety challenges in RL for autonomous driving, particularly in complex highway environments.