BAFusion: Bidirectional Attention Fusion for 3D Object Detection Based on LiDAR and Camera

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Sensor Fusion

Background:

• 3D object detection is crucial for autonomous driving and robotics.
• Conventional sensor fusion methods (LiDAR and cameras) lack flexibility and robustness due to fixed projection matrices.
• Complex environmental conditions degrade alignment accuracy in existing approaches.

Purpose of the Study:

• To propose a novel Bidirectional Attention Fusion (BAFusion) module for improved LiDAR-camera sensor fusion.
• To enhance the flexibility and robustness of 3D object detection systems.
• To address challenges in cross-modal attention calculations for sensor fusion.

Main Methods:

• Developed a Bidirectional Attention Fusion (BAFusion) module utilizing cross-attention for LiDAR and camera data.
• Introduced a Cross Focused Linear Attention Fusion (CFLAF) Layer to optimize attention complexity and inter-modal data interaction.
• Integrated the CFLAF Layer into the BAFusion pipeline for adaptive cross-modal attention weight learning.

Main Results:

• BAFusion demonstrated consistent performance improvements on the KITTI dataset across various baseline networks (PointPillars, SECOND, Part-A²).
• Significant enhancements were observed in detecting smaller objects, such as cyclists and pedestrians.
• The proposed method achieved competitive results on the KITTI benchmark, outperforming conventional fusion techniques.

Conclusions:

• The BAFusion module offers a more flexible and robust approach to 3D object detection through adaptive cross-modal attention.
• The CFLAF Layer effectively optimizes attention mechanisms and facilitates advanced sensor data interactions.
• This work presents a novel and effective solution for multi-sensor fusion challenges in autonomous systems.