Divide and Conquer: Improving Multi-Camera 3D Perception With 2D Semantic-Depth Priors and Input-Dependent Queries.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• 3D perception tasks like object detection and BEV segmentation are critical for autonomous systems.
• Existing methods struggle with integrating semantic and depth cues, leading to errors.
• Transformer models have limited capacity due to input-independent initial queries.

Purpose of the Study:

• To propose an input-aware Transformer framework (SDTR) that leverages semantic and depth priors.
• To enhance the accuracy of 3D object detection and Bird's-Eye-View segmentation.
• To address limitations in current Transformer-based 3D perception models.

Main Methods:

• Developed an S-D Encoder to explicitly model semantic and depth priors, disentangling categorization and position estimation.
• Introduced a Prior-guided Query Builder to incorporate semantic priors into initial Transformer queries for input-aware queries.
• Utilized multi-camera images as input for 3D perception tasks.

Main Results:

• Achieved state-of-the-art performance on the nuScenes and Lyft benchmarks.
• Demonstrated significant improvements in both 3D object detection and BEV segmentation.
• Validated the effectiveness of leveraging semantic and depth priors.

Conclusions:

• The proposed SDTR framework effectively integrates semantic and depth information for improved 3D perception.
• Input-aware queries and explicit prior modeling enhance Transformer performance in complex 3D tasks.
• SDTR offers a promising approach for advancing autonomous driving perception systems.