CI3D: Context Interaction for Dynamic Objects and Static Map Elements in 3D Driving Scenes.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Multi-view 3D visual perception is crucial for autonomous driving.
• Challenges exist in recovering 3D spatial information and performing context interaction from multi-view images.

Purpose of the Study:

• To propose a novel transformer-based framework, CI3D, for implicit 3D context interaction.
• To enhance 3D object detection and Birds'-eye-view (BEV) map segmentation for autonomous driving.

Main Methods:

• Utilized dynamic object queries and static map queries to gather sparse 3D spatial information from multi-view image features.
• Employed a dynamic 3D position encoder for accurate positional embeddings.
• Implemented a multi-head attention mechanism for aggregating 3D context information.
• Introduced a panoptic segmentation head and a 3D-to-2D deformable cross-attention module to address feature learning challenges.

Main Results:

• CI3D effectively models 3D context interaction between dynamic objects and static map elements.
• The proposed auxiliary tasks significantly enhance spatial feature learning robustness.
• Demonstrated superior performance on the nuScenes and Waymo datasets compared to baseline methods.

Conclusions:

• The CI3D framework offers a promising approach for improving multi-view 3D visual perception in autonomous driving.
• Implicitly modeling 3D context interaction is key to enhancing perception tasks.