Velocity control in car-following behavior with autonomous vehicles using reinforcement learning

Area of Science:

• Artificial Intelligence
• Robotics
• Transportation Engineering

Background:

• Car-following is a fundamental driving behavior crucial for autonomous vehicles (AVs) within vehicle-to-vehicle transportation systems.
• Existing car-following models often require consideration of the following vehicle, especially in mixed traffic environments.
• Ensuring safety and efficiency in AV operations necessitates robust control strategies that account for surrounding vehicles.

Purpose of the Study:

• To propose a safe velocity control method for autonomous vehicles (AVs) using reinforcement learning.
• To develop a model that considers the influence of both leading and following vehicles in a mixed traffic environment.
• To enhance the safety and efficiency of the car-following process for AVs.

Main Methods:

• A mixed driving environment simulation was created using trajectories from the naturalistic High D driving dataset.
• The soft actor-critic (SAC) algorithm was employed as the velocity control agent, with acceleration as the action.
• The state included relative distance and relative speed between the AV and surrounding vehicles; a reward function guided collision-free acceleration.

Main Results:

• The trained SAC agent demonstrated the ability to learn and execute collision avoidance maneuvers.
• Testing confirmed complete collision avoidance, achieving zero collisions in simulated scenarios.
• The SAC agent's driving performance was analyzed against human driving for safety and efficiency metrics.

Conclusions:

• Reinforcement learning, specifically the SAC algorithm, provides an effective method for safe velocity control in AVs.
• The proposed approach successfully enables AVs to avoid collisions in car-following situations with both leading and following vehicles.
• This research contributes to improving the overall safety of autonomous vehicle operations in complex traffic environments.