PORF-DDPG: Learning Personalized Autonomous Driving Behavior with Progressively Optimized Reward Function

Area of Science:

• Artificial Intelligence
• Robotics
• Machine Learning

Background:

• Deep Reinforcement Learning (DRL) shows promise for autonomous driving but struggles with complex, dynamic scenarios due to predefined reward functions.
• Ensuring safe and comfortable autonomous vehicle operation in real-world conditions remains a significant challenge for current DRL methods.

Purpose of the Study:

• To develop a human-in-the-loop DRL algorithm for learning personalized autonomous driving behaviors.
• To address the limitations of fixed reward functions in DRL for real-world driving scenarios.
• To enhance the safety and adaptability of autonomous vehicles in dynamic environments.

Main Methods:

• A novel progressively optimized reward function (PORF) learning model is proposed and integrated into the Deep Deterministic Policy Gradient (DDPG) framework, creating PORF-DDPG.
• The PORF model combines a pre-defined reward function with a Deep Neural Network (DNN) that learns driving intentions from human observers.
• The DNN-based reward component is progressively trained using front-view images and active human supervision.

Main Results:

• The proposed PORF-DDPG algorithm demonstrates online learning capabilities.
• The method shows adaptability to different environmental conditions.
• Experimental results indicate improved autonomous driving behavior learning compared to classic DRLs, especially in potentially hazardous situations.

Conclusions:

• The human-in-the-loop approach with a progressively optimized reward function enhances autonomous driving behavior learning.
• The PORF-DDPG method offers a promising solution for safe and personalized autonomous driving in complex dynamic scenarios.
• The algorithm's ability to learn from human supervision and adapt to environments suggests potential for real-world deployment.