Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving.

Area of Science:

• Computer Vision
• Robotics
• Autonomous Driving

Background:

• Bird's eye view (BEV) representations are crucial for in-vehicle 3D perception.
• Current BEV methods show promise but lack sufficient assessment of robustness in varied conditions.

Purpose of the Study:

• To introduce RoboBEV, an extensive benchmark suite for evaluating the resilience of BEV algorithms.
• To assess the impact of diverse camera corruptions and sensor failures on BEV perception models.

Main Methods:

• Developed RoboBEV benchmark with various camera corruption types and severity levels.
• Included scenarios simulating complete sensor failures for multi-modal models.
• Evaluated 33 state-of-the-art BEV models on tasks including detection, segmentation, depth, and occupancy prediction.

Main Results:

• Identified a correlation between in-distribution performance and out-of-distribution resilience.
• Demonstrated the effectiveness of pre-training and depth-free BEV transformations for robustness.
• Observed significant robustness improvements by leveraging extensive temporal information.

Conclusions:

• Insights from RoboBEV guide the development of more robust BEV models for real-world applications.
• CLIP-based strategies show promise for enhancing robustness.
• Future BEV models should prioritize both accuracy and real-world resilience.