Reinforced Refinement With Self-Aware Expansion for End-to-End Autonomous Driving

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Science

Background:

• End-to-end autonomous driving models map sensor data to driving actions.
• Existing imitation learning (IL) models struggle with generalization to difficult driving situations and lack post-deployment feedback.
• Reinforcement learning (RL) can address complex cases but often leads to overfitting and forgetting general knowledge.

Purpose of the Study:

• To introduce a novel learning pipeline, Reinforced Refinement with Self-aware Expansion (R2SE), for end-to-end autonomous driving.
• To enhance generalization to hard cases and ensure continuous improvement in driving policies.
• To overcome limitations of current IL and RL approaches in autonomous driving.

Main Methods:

• R2SE employs a three-component pipeline: Generalist Pretraining with hard-case allocation, Residual Reinforced Specialist Fine-tuning, and Self-aware Adapter Expansion.
• Generalist Pretraining identifies failure-prone cases for targeted refinement.
• Residual Reinforced Specialist Fine-tuning uses RL to optimize performance in difficult domains while preserving general knowledge.

Main Results:

• R2SE demonstrates improved generalization, safety, and long-horizon policy robustness compared to state-of-the-art end-to-end (E2E) systems.
• The method effectively refines performance in challenging driving scenarios.
• Experimental results were validated in both closed-loop simulations and real-world datasets.

Conclusions:

• Reinforced refinement offers an effective strategy for scalable autonomous driving systems.
• R2SE enables continuous improvement of driving policies by dynamically integrating specialist knowledge.
• The proposed pipeline addresses key challenges in generalization and robustness for E2E autonomous driving.