BEVFormer: Learning Bird's-Eye-View Representation From LiDAR-Camera Via Spatiotemporal Transformers

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Multi-modality fusion is crucial for competitive 3D perception in autonomous driving.
• Existing methods often lack a unified approach for integrating diverse sensor data.
• The need for effective spatiotemporal feature extraction remains a key challenge.

Purpose of the Study:

• To introduce BEVFormer, a novel framework for unified Bird's-Eye View (BEV) representation learning.
• To leverage spatiotemporal transformers for effective multi-modality data fusion.
• To support a wide array of autonomous driving perception tasks with a single model.

Main Methods:

• Developed BEVFormer, utilizing grid-shaped BEV queries to interact with spatial and temporal information.
• Implemented spatial cross-attention for fusing point cloud and camera features within the BEV space.
• Employed temporal self-attention to recurrently integrate historical BEV information.

Main Results:

• Achieved a new state-of-the-art performance with 74.1% NDS metric on the nuScenes test set.
• Demonstrated the succinctness and effectiveness of the proposed fusion method compared to other paradigms.
• Extended BEVFormer successfully to tasks including object tracking, vectorized mapping, and occupancy prediction.

Conclusions:

• BEVFormer provides a unified and effective solution for multi-modality fusion in autonomous driving perception.
• The framework's ability to handle spatiotemporal information enables superior performance across diverse tasks.
• The approach shows significant promise for advancing end-to-end autonomous driving systems.