Infrared and Visible Image Fusion via Lightweight Semantic Prior Encoding and Cross-Attention Fusion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Infrared (IR) and visible image fusion combines thermal saliency with rich textures.
• Deep learning methods exist but struggle with pixel-level operations or require segmentation supervision.
• Existing approaches limit practicality and performance in complex scenes.

Purpose of the Study:

• To develop a lightweight fusion network leveraging semantic priors without segmentation supervision.
• Introduce SPE2Fusion, a semantic prior-driven network using a dual-stage semantic injection paradigm.
• Enhance image fusion for complex scenes by integrating multi-scale scene priors.

Main Methods:

• A lightweight semantic encoder extracts multi-scale scene priors end-to-end, optimized solely by fusion loss.
• Dual-stage semantic injection: Efficient Semantic Feature Awareness (ESFA) and Efficient Semantic Feature Embedding (ESFE) modules.
• Bidirectional cross-attention fusion block integrates cross-modal features under dual semantic guidance, supervised by multi-constraint loss.

Main Results:

• SPE2Fusion achieves state-of-the-art performance on MSRS, LLVIP, and RoadScene benchmarks.
• Outperforms representative methods like CrossFuse and DDBFusion, ranking first on four of six metrics on MSRS.
• Demonstrates strong generalization on unseen datasets without domain adaptation.

Conclusions:

• SPE2Fusion effectively leverages semantic priors for advanced image fusion without segmentation masks.
• The dual-stage semantic injection paradigm enhances feature awareness and reconstruction coherence.
• The proposed method offers a practical and high-performing solution for infrared and visible image fusion.