AFTR: A Robustness Multi-Sensor Fusion Model for 3D Object Detection Based on Adaptive Fusion Transformer

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Multi-modal sensors like LiDAR and cameras are crucial for autonomous driving.
• Current fusion methods struggle with inconsistent data representations and dynamic scene misalignment.
• Existing approaches either ignore misalignment issues or incur high computational costs.

Purpose of the Study:

• To develop an end-to-end multi-sensor fusion framework that addresses the limitations of current methods.
• To improve the accuracy, efficiency, and robustness of autonomous driving systems through advanced sensor fusion.

Main Methods:

• Proposed the adaptive fusion transformer (AFTR), a transformer-based framework.
• Introduced the adaptive spatial cross-attention (ASCA) mechanism for local feature association and misalignment reduction.
• Implemented the spatial temporal self-attention (STSA) mechanism to mitigate dynamic scene displacements.

Main Results:

• Achieved state-of-the-art (SOTA) performance on the nuScenes 3D object detection task (74.9% NDS, 73.2% mAP).
• Demonstrated strong robustness to sensor misalignment, with only a 0.2% NDS drop under slight noise.
• Validated the effectiveness of individual AFTR components through ablation studies.

Conclusions:

• The proposed AFTR framework offers an accurate, efficient, and robust solution for multi-sensor data fusion.
• AFTR effectively handles challenges posed by sensor misalignment and dynamic environments in autonomous driving.
• The framework shows significant potential for advancing the reliability of self-driving systems.