Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages

Area of Science:

• Autonomous Systems
• Artificial Intelligence
• Control Theory

Background:

• Neural networks and reinforcement learning (RL) are increasingly used for autonomous vehicle control.
• The 'black box' nature of RL policies hinders trust and deployment in safety-critical applications.
• Lack of interpretability in RL control policies is a major barrier for autonomous vehicle adoption.

Purpose of the Study:

• To develop a reinforcement learning approach for autonomous vehicle longitudinal control.
• To enhance safety and interpretability using rule-based safety cages as weak supervision.
• To improve training convergence and the safety of the final control policy.

Main Methods:

• Implemented a reinforcement learning agent for autonomous vehicle longitudinal control.
• Integrated rule-based safety cages to provide weak supervision and enhance safety.
• Compared RL models with and without safety cages, and with optimal vs. constrained parameters.

Main Results:

• Weak supervision from safety cages consistently improved exploration safety, convergence speed, and model performance.
• Safety cages enabled learning a safe driving policy even when RL alone failed with constrained parameters.
• The rule-based supervisory controller offers interpretability, facilitating traditional validation and verification.

Conclusions:

• Rule-based safety cages are effective for enhancing RL-based autonomous vehicle control.
• This approach addresses the interpretability and safety challenges of deploying RL in autonomous vehicles.
• The method allows for safe policy learning even under suboptimal or constrained model conditions.