DMNet: A Frequency-Enhanced and Adaptive Spatial Fusion Network for RGB-Infrared Object Detection

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Object detection in challenging environments (illumination variations, clutter, small objects) is difficult.
• Multimodal detection using RGB and infrared (IR) data offers complementary information but faces challenges like feature misalignment and detail loss.
• Existing methods lack sufficient semantic interaction for robust multimodal object detection.

Purpose of the Study:

• To introduce DMNet, a novel dual-stream framework for enhanced visible and IR multimodal object detection.
• To address cross-modal feature misalignment, loss of fine-grained details, and insufficient semantic interaction in existing methods.
• To develop an efficient and effective solution for object detection in complex, low-light, and small-object scenarios.

Main Methods:

• Developed a dual-stream framework, DMNet, integrating Surface Detail Fusion (SDF), Wavelet Feature Extraction (WFE), Context-Guided Enhancement (CGE), and Adaptive Spatial Fusion (ASF).
• SDF aligns shallow features, WFE enhances frequency-domain information, CGE refines semantics, and ASF aggregates multi-scale features.
• Evaluated DMNet on M3FD, LLVIP, and VEDAI benchmark datasets.

Main Results:

• DMNet achieved superior detection performance compared to existing methods across three benchmark datasets.
• Achieved mAP@0.5 scores of 78.4% on M3FD, 94.4% on LLVIP, and 59.0% on VEDAI.
• The model demonstrates high efficiency with only 5.72 million parameters, suitable for practical deployment.

Conclusions:

• DMNet effectively overcomes challenges in visible and IR multimodal object detection, particularly in low-light and small-object scenarios.
• The proposed framework offers a significant improvement in detection accuracy and efficiency.
• DMNet presents a practical and high-performing solution for complex object detection tasks.