MCF-YOLO: Consistency-Guided Cross-Modal Attention for Small-Object RGB-IR Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Fusion

Background:

• Single-modality RGB detectors struggle in low-light, occluded, and cluttered environments.
• Infrared (IR) imaging offers stable visibility but lacks texture/color and is affected by noise.

Purpose of the Study:

• To develop an RGB-IR object detection network (MCF-YOLO) that overcomes limitations of single-modality detectors.
• To improve detection accuracy and robustness in challenging visual conditions.

Main Methods:

• Proposing the MCF-YOLO network with three core components: Cross-Modal Hierarchical Fusion (CMHF), Soft Attention Regularization based on Attention Prior (SAR-AP), and Small-Object-Sensitive Detection Head (SOS-Head).
• CMHF module aligns and fuses multi-scale RGB and IR features.
• SAR-AP uses IR features to guide cross-modal attention, reducing noise impact.
• SOS-Head enhances small target detection using high-resolution features.

Main Results:

• MCF-YOLO achieved significant improvements on M3FD and VEDAI benchmarks.
• Demonstrated gains of 2.7% mAP@0.5 and 1.1% mAP@0.5:0.95 on M3FD.
• Showcased gains of 5.4% mAP@0.5 and 4.4% mAP@0.5:0.95 on VEDAI.

Conclusions:

• Consistency-guided cross-modal fusion enhances RGB-IR object detection.
• High-resolution small-target modeling is crucial for long-range and occluded scenarios.
• MCF-YOLO offers a robust solution for object detection in low-visibility and cluttered scenes.