Constraint-Guided Behavior Transformer for Centralized Coordination of Connected and Automated Vehicles at...

Area of Science:

• Intelligent Transportation Systems
• Artificial Intelligence in Transportation
• Autonomous Driving Systems

Background:

• Centralized coordination of Connected and Automated Vehicles (CAVs) at unsignalized intersections is crucial for traffic efficiency, safety, and comfort.
• Existing Autonomous Intersection Management (AIM) systems, including rule-based, optimization, and current Reinforcement Learning (RL) methods, face challenges in generalization, computational efficiency, policy inference, and safety.
• Complex traffic environments and dynamic conditions necessitate advanced coordination strategies.

Purpose of the Study:

• To propose a novel safe Reinforcement Learning (RL) method, Constraint-Guided Behavior Transformer for Safe Reinforcement Learning (CoBT-SRL), for centralized coordination of CAVs at unsignalized intersections.
• To enhance decision-making efficiency and safety in autonomous intersection management.
• To improve upon existing RL-based and optimal control methods for CAV coordination.

Main Methods:

• Utilizes transformers as the policy network to capture long-range dependencies and improve data sample efficiency by learning from historical states, actions, and returns.
• Incorporates a sequence-level entropy regularizer to boost policy exploration while maintaining safety during policy updates.
• Employs a constraint-guided approach within the behavior transformer framework for safe RL.

Main Results:

• CoBT-SRL demonstrates stable training progress and effective convergence in simulations.
• The proposed method outperforms other RL methods and Vehicle Intersection Coordination Schemes (VICS) based on optimal control.
• Significant improvements were observed in traffic efficiency, driving safety, and passenger comfort.

Conclusions:

• CoBT-SRL offers an effective solution for safe and efficient autonomous intersection management.
• The transformer-based approach with sequence-level entropy regularization enhances RL performance for CAV coordination.
• This method represents a significant advancement in ensuring safety and efficiency for connected and automated vehicles in complex traffic scenarios.