3D car-detection based on a Mobile Deep Sensor Fusion Model and real-scene applications

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Accurate automotive perception is crucial for unmanned vehicles, especially in complex environments.
• Single-sensor systems struggle with reliable car detection, necessitating multi-sensor fusion.
• Deep learning advancements offer new possibilities for sensor fusion in automotive perception.

Purpose of the Study:

• To propose and evaluate a novel Mobile Deep Sensor Fusion Model (MDSFM) for robust car detection.
• To enhance the accuracy and generalization ability of car detection systems in challenging conditions.
• To validate the practical applicability of the proposed model in real-world driverless scenarios.

Main Methods:

• Developed MDSFM integrating LiDAR and camera data using deep learning.
• Implemented a revised squeezeNet for LiDAR processing and an improved R-CNN with Mobile Spatial Attention Module (MSAM) for camera processing.
• Utilized a dual-view deep fusing structure and projected 3D data to 2D for efficient training.
• Validated the model on KITTI datasets and implemented a ROS program on an experimental car.

Main Results:

• MDSFM demonstrated significantly improved performance in detecting vehicles, especially in complex environments.
• The model enhanced data quality, improved generalization ability, and preserved contextual relevance.
• Achieved stable performance in simulated driverless environments and proved effective in realistic scenarios.

Conclusions:

• MDSFM offers a robust and efficient solution for multi-sensor fusion in automotive perception.
• The proposed model shows significant practical value and potential for enhancing the safety and reliability of autonomous driving systems.
• The fusion of deep learning with multi-sensor data is a promising direction for advancing perception capabilities in unmanned vehicles.