Graph-DETR4D: Spatio-Temporal Graph Modeling for Multi-View 3D Object Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Multi-View 3D object detection (MV3D) leverages multiple cameras but struggles with monocular depth estimation challenges.
• Existing methods like Graph-DETR3D improve 3D object detection but have limitations in depth and velocity estimation due to single-frame inputs.

Purpose of the Study:

• To develop a unified spatial-temporal graph modeling framework for enhanced multi-view 3D object detection.
• To improve depth and velocity estimation accuracy by utilizing multi-frame inputs and a dynamic 4D graph architecture.
• To create a computationally efficient MV3D framework suitable for real-world applications through knowledge distillation.

Main Methods:

• Introduced a unified spatial-temporal graph modeling framework operating in 4D space, integrating multi-frame inputs.
• Employed an attention mechanism within the dynamic graph to capture both spatial and temporal image cues.
• Proposed a dense-sparse distillation framework to reduce computational load without compromising detection accuracy.

Main Results:

• The proposed Graph-DETR4D framework demonstrates significant improvements in multi-view 3D object detection.
• Achieved a competitive 62.0% NDS on the nuScenes test leaderboard, showcasing effectiveness and efficiency.
• The distillation method successfully reduces computational cost, making the model more practical for deployment.

Conclusions:

• Graph-DETR4D offers a faster and stronger solution for multi-view 3D object detection by effectively integrating spatial-temporal information.
• The framework addresses limitations in depth and velocity estimation, outperforming previous single-frame approaches.
• The developed distillation technique enhances the model's efficiency for real-world autonomous driving systems.