Multimodal road perception with illumination adaptation in autonomous vehicles

Area of Science:

• Autonomous Driving Systems
• Computer Vision and Sensor Fusion
• Artificial Intelligence for Transportation

Background:

• Autonomous driving perception falters in low-light (rain, fog, night).
• Existing multimodal fusion methods are static and struggle with varying illumination.
• Robust road passability assessment is crucial for safe autonomous navigation.

Purpose of the Study:

• To develop an illumination-adaptive multimodal perception framework for autonomous driving.
• To enhance perception robustness in adverse low-light conditions.
• To improve safety-critical region recognition and road passability assessment.

Main Methods:

• Proposed SafeDrive-Fusion: an illumination-adaptive multimodal perception framework.
• Implemented dynamic cross-modal attention fusion with safety-critical region prioritization.
• Introduced an adaptive weighting mechanism to dynamically integrate visual and radar data.

Main Results:

• Maintained 87.5% accuracy in extremely low-light conditions (6.1% degradation).
• Achieved an F1 score of 0.82 for hazardous situation recognition (0.40 higher than baselines).
• Demonstrated dynamic modal weight adjustment: radar weight increased from 25% to 62% as illumination decreased.

Conclusions:

• SafeDrive-Fusion significantly improves autonomous driving perception in low-light environments.
• The adaptive fusion and prioritization framework enhances safety and reliability.
• The system meets real-time requirements with a 45ms latency.