SDEF-BEV: spatial-aware dual-expert radar-camera fusion for robust BEV 3D object detection

Area of Science:

• Computer Vision
• Autonomous Driving Systems
• Sensor Fusion

Background:

• Bird's-eye view (BEV) radar-camera fusion is crucial for 3D perception in autonomous vehicles.
• Existing fusion methods face challenges with spatial misalignment and dynamic adaptation of sensor contributions.

Purpose of the Study:

• To propose SDEF-BEV, a novel spatial-aware dual-expert fusion network for robust 3D perception.
• To address limitations in current radar-camera fusion techniques regarding spatial alignment and adaptive fusion.

Main Methods:

• Introduced a parallel dual-path fusion architecture: Spatial-Aware Dual-Expert Fusion (SDEF) and Channel and Spatial Fusion (CSF).
• The SDEF module uses specialized experts and a spatial gating network for adaptive, location-wise fusion weights.
• The CSF path leverages convolutional inductive biases to maintain spatial context.

Main Results:

• SDEF-BEV achieved competitive performance on the nuScenes dataset with an NDS of 57.1% and mAP of 45.7%.
• Ablation studies confirmed the effectiveness of the parallel SDEF architecture.
• The method demonstrated strong robustness, especially in adverse weather conditions.

Conclusions:

• SDEF-BEV offers an effective solution for spatial-aware radar-camera fusion in autonomous driving.
• The proposed adaptive fusion strategy enhances 3D perception accuracy and robustness.
• The network shows significant potential for real-world autonomous driving applications.