RTMF-Net: A Dual-Modal Feature-Aware Fusion Network for Dense Forest Object Detection

Area of Science:

• Remote Sensing
• Computer Vision
• Artificial Intelligence

Background:

• Multimodal remote sensing object detection leverages complementary RGB and thermal infrared (TIR) data.
• Existing methods are often computationally intensive, hindering real-time applications.
• There is a need for efficient frameworks balancing accuracy and speed.

Purpose of the Study:

• To propose a lightweight and efficient multimodal object detection framework (RTMF-Net) for real-time remote sensing.
• To introduce novel backbone architectures and fusion mechanisms for improved performance.
• To enhance cross-modal feature interaction and localization accuracy.

Main Methods:

• RTMF-Net employs a dual-stream structure with modality-specific enhancement modules (CEFBlock for RGB, DLEBlock for TIR).
• A Weighted Denoising Fusion Module with Enhanced Fusion Attention (EFA) is used for adaptive feature fusion.
• A Shape-Aware Intersection over Union (SA-IoU) loss function improves localization robustness.

Main Results:

• RTMF-Net achieved competitive mAP scores of 98.7% (ODinMJ) and 95.7% (LLVIP).
• The framework is lightweight, with only 4.3M parameters and 11.6 GFLOPs.
• Demonstrated a favorable balance between detection accuracy and computational efficiency.

Conclusions:

• RTMF-Net effectively addresses the limitations of existing methods for real-time multimodal object detection.
• The proposed enhancements in backbone and fusion strategies significantly improve performance.
• RTMF-Net is suitable for real-time remote sensing applications requiring efficient and accurate object detection.