JoVE - Journal of Visualized Experiments
JoVE Visualize
Contact Us

Text-Guided Semantic Alignment Network With Spatial-Frequency Interaction for Infrared-Visible Image Fusion Under Extreme Illumination

IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society +

|

November 25, 2025

|

November 25, 2025

View abstract on PubMed

Summary

This summary is machine-generated.

This study introduces TSANet, a novel network for infrared-visible image fusion under extreme illumination. TSANet effectively aligns semantic features, significantly improving fusion performance and visual quality.

Area Of Science

  • Computer Vision
  • Artificial Intelligence
  • Image Processing

Background

  • Existing infrared-visible image fusion methods struggle with extreme illumination due to ignored semantic differences.
  • Limited improvement is observed when textual and visual features are not properly aligned.

Purpose Of The Study

  • To propose a novel Text-Guided Semantic Alignment Network (TSANet) for enhanced infrared-visible image fusion.
  • To address the challenge of semantic misalignment between textual and visual features in extreme illumination conditions.

Main Methods

  • TSANet employs an encoder-decoder structure with parallel image and text encoders.
  • A Semantic Alignment and Fusion (SAF) block integrates Semantic Alignment (SA) and Spatial-Frequency Interaction (SFI) modules.
  • SA modules align visual features with corresponding textual features, while SFI modules aggregate spatial-frequency information.

Main Results

  • Extensive experiments demonstrate TSANet's effectiveness and superiority over seven state-of-the-art methods.
  • The proposed method achieves a visually pleasing fusion effect by progressively complementing image modalities.
  • A new dataset for extreme-illumination image fusion was created and utilized.

Conclusions

  • TSANet effectively overcomes limitations of existing methods by incorporating semantic alignment.
  • The network achieves superior performance in infrared-visible image fusion under challenging illumination.
  • The developed dataset and source code facilitate further research in this domain.

Related Concept Videos

Infrared (IR) Spectroscopy: Overview 01:09

4.6K

When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...

IR Frequency Region: Fingerprint Region 01:03

1.8K

IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...

Insensitive Nuclei Enhanced by Polarization Transfer (INEPT) 01:15

973

Insensitive Nuclei Enhanced by Polarization Transfer (INEPT) is an advanced Nuclear Magnetic Resonance (NMR) technique specifically designed to detect and enhance the signals of low-abundance nuclei, such as carbon-13 and nitrogen-15, in small molecules. The fundamental principle behind INEPT is the transfer of polarization from a more abundant and highly polarizable nucleus, typically hydrogen-1, to the low-abundance nucleus of interest. This process effectively boosts the NMR signal of the...

Super-resolution Fluorescence Microscopy 01:37

12.1K

Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...

Imaging Biological Samples with Optical Microscopy 01:18

8.8K

Optical microscopy uses optic principles to provide detailed images of samples. Antonie van Leeuwenhoek designed the first compound optical microscope in the 17th century to visualize blood cells, bacteria, and yeast cells. In 1830, Joseph Jackson Lister created an essentially modern light microscope. The 20th century saw the development of microscopes with enhanced magnification and resolution.
In optical microscopy, the specimen to be viewed is placed on a glass slide and clipped on the stage...