Robust Infrared-Visible Fusion Imaging with Decoupled Semantic Segmentation Network.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Infrared and visible image fusion offers complementary data for surveillance and military applications.
• Existing fusion methods often lack objective evaluation and fail to address low-light conditions effectively.
• High-level visual task performance is often independent of subjective fusion quality metrics.

Purpose of the Study:

• To develop a novel decoupled and semantic segmentation-driven network for infrared and visible image fusion.
• To improve fusion quality and performance in challenging low-light scenarios.
• To enhance target detection and overall accuracy in maritime environments.

Main Methods:

• A cross-modality transformer fusion module for hierarchical feature learning.
• A semantic-driven fusion module to emphasize prominent target features.
• A weighted fusion strategy and a refined loss function for natural fusion images.

Main Results:

• The proposed method achieves superior fusion quality metrics on public and a new Maritime Infrared and Visible (MIV) dataset.
• Achieved over 96% target detection accuracy and a high mAP@[50:95] value.
• Demonstrated robustness and generalization in practical maritime environmental perception tasks.

Conclusions:

• The semantic segmentation-driven approach effectively links fusion with downstream tasks.
• The method significantly outperforms existing approaches in both objective metrics and practical applications.
• The new MIV dataset facilitates evaluation for maritime surveillance.