SparseBEV: A Fully Sparse Framework for Multi-View 3D Object Detection.

Area of Science:

• Computer Vision
• Autonomous Driving Systems
• Machine Learning

Background:

• Camera-based 3D object detection in Bird's Eye View (BEV) space is crucial for autonomous systems.
• Dense detectors offer high accuracy but are computationally intensive and complex.
• Sparse detectors are faster but historically underperform compared to dense methods.

Purpose of the Study:

• To develop a fully sparse 3D object detector that matches or exceeds the performance of dense detectors while offering improved speed.
• To enhance the adaptability of sparse detectors in both BEV and image spaces for better feature aggregation and interaction.

Main Methods:

• Proposed a fully sparse 3D object detector with three key innovations: scale-adaptive self-attention in BEV, scale-adaptive cross-attention for temporal dynamics, and adaptive sampling/mixing for query-image feature interaction.
• Introduced two temporal modeling approaches: sampling-point-based multi-frame stacking (SparseBEV) and query-based recurrent temporal fusion (SparseBEV++).
• Evaluated the models on the nuScenes and Waymo datasets.

Main Results:

• SparseBEV and SparseBEV++ outperformed all previous methods on the nuScenes dataset.
• SparseBEV achieved 55.8 NDS at 23.5 FPS, while SparseBEV++ reached 57.1 NDS at 24.6 FPS.
• On the Waymo dataset, SparseBEV++ achieved 58.9 mAP and 55.2 mAPH, surpassing prior methods.

Conclusions:

• The proposed fully sparse 3D object detector effectively bridges the performance gap with dense detectors.
• The key designs significantly enhance detector adaptability, leading to state-of-the-art results in 3D object detection.
• SparseBEV++ demonstrates a promising balance of high accuracy and real-time inference speed for autonomous driving applications.