Object Detection in Multispectral Remote Sensing Images Based on Cross-Modal Cross-Attention.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Sensor Fusion

Background:

• Single visible images are insufficient for perception in complex environments.
• Target detection in extreme conditions like nighttime and fog presents significant challenges.

Purpose of the Study:

• To develop a novel dual-stream real-time detector for target detection in extreme environments.
• To efficiently utilize both visible and infrared images for enhanced environmental sensing.

Main Methods:

• Expanded YOLOv8 backbone into a parallel dual-stream architecture for simultaneous multi-modal processing.
• Introduced a cross-modal feature enhancement module using attention mechanisms to prevent information loss and improve small target detection.
• Proposed a three-stage fusion strategy (spatial, channel, overall dimensions) for optimal feature integration.

Main Results:

• Achieved state-of-the-art performance in detecting small targets across two datasets.
• Demonstrated excellent stability and robust detection capabilities in challenging environmental conditions.
• Validated the effectiveness of the end-to-end trained cross-modal feature fusion module.

Conclusions:

• The proposed Fast All-Weather environment sensing (FAWDet) detector offers a new solution for target detection in extreme environments.
• FAWDet provides stable and robust performance, outperforming existing methods, especially for small targets.
• This dual-stream approach effectively fuses visible and infrared data for comprehensive environmental perception.