Adaptive Cruise Control Based on Safe Deep Reinforcement Learning.

Area of Science:

• Automotive Engineering
• Artificial Intelligence
• Control Systems

Background:

• Current adaptive cruise control (ACC) methods face challenges with modeling difficulties and computational efficiency.
• Optimization control-based ACC systems often struggle to balance performance with safety and comfort.

Purpose of the Study:

• To propose a novel adaptive cruise control system, Safety-First Reinforcement Learning Adaptive Cruise Control (SFRL-ACC), leveraging Deep Reinforcement Learning (DRL).
• To overcome the limitations of existing ACC methods by improving computational efficiency, safety, and ride comfort.

Main Methods:

• Formulated the ACC problem as a safe DRL problem using a Constrained Markov Decision Process (CMDP).
• Developed the Projected Constrained Policy Optimization (PCPO)-based ACC Algorithm (SFRL-ACC) to solve the CMDP.
• Incorporated safety constraints into the DRL policy updates using Kullback-Leibler (KL) divergence trust regions.

Main Results:

• The SFRL-ACC system demonstrated superior performance compared to state-of-the-art Model Predictive Control (MPC)-based ACC methods.
• Experimental results showed improvements in computation time, traffic efficiency, ride comfort, and safety.
• The PCPO algorithm effectively maximized performance while adhering to safety cost bounds.

Conclusions:

• SFRL-ACC offers a more efficient, safer, and comfortable alternative to current ACC systems.
• The proposed safe DRL approach effectively addresses the complexities of ACC control.
• This research highlights the potential of DRL in developing advanced automotive control systems.