S4Fusion: Saliency-Aware Selective State Space Model for Infrared and Visible Image Fusion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Multi-modal image fusion is vital for vision tasks, but current methods struggle to preserve complementary features.
• Convolutional Neural Networks (CNNs) use local fields, while Transformers lack comprehensive spatial interaction, leading to modality bias in fused images.

Purpose of the Study:

• To propose a novel fusion framework, S4Fusion (Saliency-aware Selective State Space), to effectively preserve complementary features and salient targets from multi-modal inputs.
• To address the limitations of existing methods in capturing global spatial information and cross-modal interactions.

Main Methods:

• Introduction of the Cross-Modal Spatial Awareness Module (CMSA) to capture global spatial information and promote cross-modal interaction.
• Development of a novel perception-enhanced loss function to guide adaptive preservation of salient objects by minimizing ambiguity in fused images.

Main Results:

• S4Fusion enables a more comprehensive representation of complementary features by effectively integrating global spatial information.
• The proposed framework demonstrates improved ability to adaptively preserve salient targets from all input modalities, reducing modality bias.

Conclusions:

• S4Fusion offers a significant advancement in multi-modal image fusion by enhancing feature preservation and saliency retention.
• The framework shows promise for improving downstream vision tasks that rely on accurate and comprehensive fused image representations.