T360Fusion: Temporal 360 Multimodal Fusion for 3D Object Detection via Transformers

Area of Science:

• Robotics and Autonomous Systems
• Computer Vision
• Sensor Fusion

Background:

• Object detection is crucial for autonomous driving, utilizing sensors like LiDAR, RGB, and thermal cameras.
• Individual sensors have limitations: RGB cameras struggle in poor lighting, while LiDAR lacks high spatial resolution.
• Thermal cameras offer robust perception in low visibility, complementing other sensors.

Purpose of the Study:

• To propose a novel multimodal sensor fusion method for autonomous driving.
• To integrate LiDAR, 360 RGB, and 360 thermal cameras to overcome individual sensor limitations.
• To enhance object detection accuracy, spatial coverage, and robustness.

Main Methods:

• A feature-level fusion strategy is employed.
• Multiple LiDAR frames are temporally accumulated and synchronized.
• Integration of 360-degree RGB and thermal imaging data.

Main Results:

• The proposed method significantly improves object detection accuracy.
• Enhanced spatial coverage and robustness are achieved through sensor fusion.
• Comprehensive environmental awareness is provided, reducing blind spots.

Conclusions:

• Multimodal sensor fusion effectively addresses individual sensor limitations in autonomous driving.
• The novel fusion method enhances perception capabilities, particularly in complex and dynamic scenes.
• This approach contributes to safer and more reliable autonomous navigation.