Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor.

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion
• Machine Learning

Background:

• Accurate obstacle detection is critical for autonomous driving safety.
• Existing sensor fusion methods often struggle with the sparsity of radar data.
• Integrating mmWave radar and vision sensors offers complementary strengths for perception.

Purpose of the Study:

• To propose a novel spatial attention fusion (SAF) method for obstacle detection.
• To effectively fuse sparse mmWave radar data with vision sensor features.
• To enhance the performance of deep learning-based object detection frameworks for autonomous driving.

Main Methods:

• Developed a spatial attention fusion (SAF) method that considers radar point sparsity.
• Integrated SAF into the feature-extraction stage, generating an attention weight matrix for fusion.
• Built a generation model to convert sparse radar points into radar images for neural network training.
• Enabled end-to-end training of the SAF method within a deep learning object detection framework.

Main Results:

• The proposed SAF method achieves superior performance in obstacle detection compared to existing fusion techniques.
• Demonstrated effective fusion of mmWave radar and vision sensor features, addressing radar data sparsity.
• Achieved state-of-the-art results on public benchmarking datasets.

Conclusions:

• The spatial attention fusion (SAF) method significantly improves obstacle detection accuracy for autonomous driving.
• SAF offers a robust approach to sensor fusion, particularly for sparse radar data.
• The developed method is compatible with deep learning object detection frameworks and will be open-sourced.