A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Infrared and visible image fusion is crucial for enhancing image quality and information extraction.
• Existing methods often struggle to balance the preservation of low-level features and high-level semantic information.
• Semantic information can improve the accuracy and detail of fused images.

Purpose of the Study:

• To propose a novel generative adversarial network (GAN) for infrared and visible image fusion that incorporates semantic segmentation.
• To enhance the fusion process by considering both low-level features and high-level semantic information.
• To improve the preservation of thermal targets and texture details in the fused images.

Main Methods:

• A generative adversarial network (GAN) named SSGAN is proposed, utilizing semantic segmentation (SS).
• The generator employs a dual-encoder-single-decoder architecture to extract distinct features for foregrounds and backgrounds.
• The discriminator's input is strategically designed using semantic masks, combining infrared foregrounds and visible backgrounds.

Main Results:

• The SSGAN effectively preserves the prominence of thermal targets from infrared images and texture details from visible images.
• Qualitative and quantitative experiments demonstrate superior performance compared to state-of-the-art fusion methods.
• The proposed method achieves a better balance between low-level and high-level feature preservation.

Conclusions:

• The proposed SSGAN offers a significant advancement in infrared and visible image fusion by leveraging semantic information.
• This approach enhances the quality and interpretability of fused images for various applications.
• The method provides a robust framework for image fusion tasks requiring detailed feature representation.