One-Stage Multi-Sensor Data Fusion Convolutional Neural Network for 3D Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• 3D object detection is crucial for robotics, autonomous driving, and automated systems.
• Multi-sensor data (e.g., Lidar, cameras) offers rich information for improved detection.
• Existing methods may not fully leverage multimodal sensor data for optimal performance.

Purpose of the Study:

• To propose a novel Convolution Neural Network (CNN) for effective 3D object detection using multi-sensor data fusion.
• To enhance the accuracy and efficiency of 3D object detection systems.
• To introduce the Complex-Retina network for synchronous feature extraction and data fusion.

Main Methods:

• Developed a unified architecture with dual feature extraction networks for synchronous processing of point cloud and image data.
• Implemented a 3D anchor projection and fusion mechanism with 2D anchors.
• Utilized multipath fully connected layers for object classification and 3D bounding box regression.

Main Results:

• The Complex-Retina network demonstrated superior performance compared to existing algorithms on the KITTI dataset.
• Achieved higher average precision (AP) and reduced time consumption.
• Successfully balanced detection accuracy and processing speed, indicating a one-stage CNN approach.

Conclusions:

• The proposed Complex-Retina network is effective for 3D object detection using multi-sensor fusion.
• The method offers a significant improvement in both accuracy and efficiency for real-world applications.
• The network provides a viable solution for scenarios requiring fast and precise 3D object detection.