GEM-YOLO: A Lightweight and Real-Time RGBT Object Detector with Gated Multimodal Fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Edge Computing

Background:

• Robust all-weather perception relies on Red-Green-Blue-Thermal (RGBT) object detection.
• Deploying RGBT detectors on edge devices faces challenges like inefficient multimodal fusion, feature loss in small objects, and high computational demands.

Purpose of the Study:

• To develop a real-time, lightweight RGBT detector for resource-constrained edge devices.
• To improve adaptive multimodal fusion, preserve small-object features, and reduce computational overhead.

Main Methods:

• Proposed GEM-YOLO, featuring an Adaptive Multimodal Gated Fusion Mechanism (GFM) for dynamic weight calibration and noise suppression.
• Integrated Space-to-Depth (SPD) convolutions for lossless downsampling to prevent small target feature collapse.
• Utilized a lightweight Ghost-Neck with Ghost modules and GSConv to minimize computational redundancy.

Main Results:

• GEM-YOLO achieved 7.58 GFLOPs and 3.44M parameters, reducing computational cost by 18.6% compared to the YOLOv11n baseline.
• Attained competitive mAP@50 scores of 82.8% on FLIR and 69.0% on M3FD datasets.
• Demonstrated superior performance in small-target localization while maintaining low computational overhead.

Conclusions:

• GEM-YOLO provides an effective solution for real-time RGBT object detection on edge platforms.
• The proposed methods enhance fusion adaptivity, preserve small object features, and reduce computational load.
• This lightweight detector is promising for practical multispectral perception in resource-limited environments.