Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Existing infrared-visible image fusion methods struggle with extreme illumination due to ignored semantic differences.
• Limited improvement is observed when textual and visual features are not properly aligned.

Purpose of the Study:

• To propose a novel Text-Guided Semantic Alignment Network (TSANet) for enhanced infrared-visible image fusion.
• To address the challenge of semantic misalignment between textual and visual features in extreme illumination conditions.

Main Methods:

• TSANet employs an encoder-decoder structure with parallel image and text encoders.
• A Semantic Alignment and Fusion (SAF) block integrates Semantic Alignment (SA) and Spatial-Frequency Interaction (SFI) modules.
• SA modules align visual features with corresponding textual features, while SFI modules aggregate spatial-frequency information.

Main Results:

• Extensive experiments demonstrate TSANet's effectiveness and superiority over seven state-of-the-art methods.
• The proposed method achieves a visually pleasing fusion effect by progressively complementing image modalities.
• A new dataset for extreme-illumination image fusion was created and utilized.

Conclusions:

• TSANet effectively overcomes limitations of existing methods by incorporating semantic alignment.
• The network achieves superior performance in infrared-visible image fusion under challenging illumination.
• The developed dataset and source code facilitate further research in this domain.