A Two-Phase Cross-Modality Fusion Network for Robust 3D Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• 3D object detection is crucial for autonomous systems.
• Integrating RGB images and LiDAR point clouds offers complementary data for enhanced perception.
• Existing methods often face challenges in achieving both high precision and robustness.

Purpose of the Study:

• To propose a robust and high-precision two-phase cross-modality fusion detector for 3D object detection.
• To leverage the strengths of both RGB images and LiDAR intensity maps for improved feature representation.
• To achieve state-of-the-art performance on benchmark datasets with reduced computational cost.

Main Methods:

• A two-stream fusion network within the Faster RCNN framework for 2D detection using RGB images and LiDAR intensity maps.
• A multi-layer feature-level fusion scheme to enhance multi-modal feature expressiveness.
• A decision-level fusion approach involving 3D frustum generation for a second-phase 3D detector, enabling instance segmentation and 3D-box regression.

Main Results:

• Features from RGB images and LiDAR intensity maps effectively complement each other.
• The proposed detector achieved state-of-the-art performance on the KITTI benchmark.
• The method demonstrated a substantially lower running time compared to existing competitors.

Conclusions:

• The two-phase cross-modality fusion approach significantly enhances 3D object detection accuracy and robustness.
• The integration of RGB and LiDAR intensity data provides a powerful combination for perception tasks.
• The proposed detector offers a computationally efficient solution for real-time applications.