Image fusion using Y-net-based extractor and global-local discriminator.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Deep learning image fusion methods face challenges in extracting and preserving information-rich features from diverse source images.
• Existing techniques often struggle with minimizing distortions in the fused output.

Purpose of the Study:

• To develop an advanced Generative Adversarial Network (GAN)-based scheme for infrared and visible image fusion.
• To improve the extraction and preservation of multi-scale features for enhanced fused image quality.

Main Methods:

• Utilized a Y-Net architecture as the generator backbone, incorporating residual dense blocks (RDblocks) for multi-scale representation learning.
• Implemented cross-modality shortcuts with contextual attention (CMSCA) for selective feature aggregation.
• Employed a unified global-local discriminator architecture (combining global GAN and PatchGAN) for detailed difference detection.

Main Results:

• The proposed method effectively learns discriminative multi-scale representations, leading to more realistic fused images.
• CMSCA facilitated the construction of information-rich fused images with improved visual effects.
• The global-local discriminator enhanced the generator's ability to capture both local radiation and global details, achieving fusion without explicit rules.

Conclusions:

• The developed GAN-based fusion scheme demonstrates superior performance in meaningful information preservation compared to state-of-the-art methods.
• The integration of multi-scale feature extraction and a global-local discriminator offers a robust solution for infrared and visible image fusion.