RoLiC: A Robust LiDAR-Camera Fusion Framework for 3D Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• LiDAR and camera fusion are crucial for 3D object detection in autonomous driving.
• Sensor failures in real-world scenarios pose significant risks to fusion-based detection systems.

Purpose of the Study:

• To develop a unified framework, RoLiC, for robust LiDAR-camera fusion that addresses LiDAR failure, camera failure, and simultaneous sensor failure.
• To enhance the reliability and effectiveness of 3D object detection under partial or complete sensor data loss.

Main Methods:

• Introduced cross-modality feature transformers (L2C and C2L) for bidirectional feature completion.
• Designed a Sparse Similarity Loss (SSL) to improve feature learning in high-probability object regions.
• Integrated a task-aware two-stage feature knowledge distillation strategy (MS1 and MS2) for cross-modality learning.

Main Results:

• RoLiC demonstrated superior performance across all tested sensor-failure scenarios on nuScenes and KITTI benchmarks.
• The framework effectively mitigates cross-modal dependency and recovers missing information.
• Outperformed state-of-the-art methods in robust 3D object detection under challenging conditions.

Conclusions:

• RoLiC provides a robust and unified solution for LiDAR-camera fusion in autonomous driving, effectively handling sensor failures.
• The proposed methods significantly improve detection reliability and performance when one or both sensors fail.
• This framework enhances the safety and dependability of autonomous driving systems.