A DINO-based progressive semantic enhanced infrared and visible image fusion network

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Existing infrared and visible image fusion methods often neglect semantic information crucial for downstream tasks.
• Current semantic-driven approaches are limited by reliance on labelled data with restricted semantic targets.

Purpose of the Study:

• To propose a novel DINO-based progressive semantic enhanced infrared and visible image fusion network (DPSEF).
• To leverage self-supervised learning for enhanced semantic feature extraction and integration in image fusion.

Main Methods:

• Utilizing the DINO (self-supervised model) to extract fine-grained spatial semantic features from unlabelled images.
• Introducing a semantic enhanced fusion module (SEFM) to progressively inject semantic priors into the fusion network.
• Developing a progressive fusion mechanism to guide the model towards target-relevant regions.

Main Results:

• DPSEF significantly surpasses mainstream algorithms in fused image visual quality.
• Demonstrated strong potential for high-level vision applications through qualitative and quantitative analyses.
• Validated the network's generality and robustness on multi-focus image fusion tasks.

Conclusions:

• The proposed DPSEF network effectively integrates rich semantic and detailed information for high-quality image fusion.
• DPSEF addresses limitations of existing methods by utilizing unlabeled data for semantic enhancement.
• The network shows significant promise for advancing computer vision applications requiring semantically rich fused imagery.