Fully Sparse Fusion for 3D Object Detection.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Current multi-modal 3D detection methods often use dense Bird's-Eye-View (BEV) feature maps, which are computationally expensive and not scalable for long-range detection.
• LiDAR-only fully sparse architectures offer efficiency for long-range perception but lack multi-modal integration.
• There is a need for scalable and efficient multi-modal 3D detection methods capable of long-range perception.

Purpose of the Study:

• To develop a multi-modal fully sparse detector that overcomes the limitations of dense detectors and LiDAR-only sparse methods.
• To enhance long-range perception capabilities in 3D object detection by integrating 2D instance segmentation.
• To achieve state-of-the-art performance and improved efficiency in multi-modal 3D detection.

Main Methods:

• Proposed an instance-based fusion framework that integrates 2D instance segmentation into the LiDAR processing pipeline.
• Maintained full sparsity throughout the multi-modal detection framework.
• Leveraged a LiDAR-only fully sparse architecture as a baseline and enhanced it with multi-modal fusion.

Main Results:

• Achieved state-of-the-art performance on the nuScenes, Waymo Open Dataset, and Argoverse 2 datasets.
• Demonstrated significantly improved inference speed, being 2.7x faster than other state-of-the-art multi-modal 3D detection methods in long-range settings.
• The proposed framework successfully maintained full sparsity while enhancing detection capabilities.

Conclusions:

• The developed instance-based fusion framework offers an efficient and scalable solution for multi-modal 3D object detection, particularly for long-range scenarios.
• This approach effectively combines the strengths of sparse architectures and multi-modal data for superior performance.
• The method represents a significant advancement in efficient and accurate 3D perception for autonomous systems.