HDCTfusion: Hybrid Dual-Branch Network Based on CNN and Transformer for Infrared and Visible Image Fusion.

Area of Science:

• Computer Vision
• Image Processing
• Artificial Intelligence

Background:

• Infrared and visible image fusion aims to integrate complementary information from both modalities.
• Existing fusion methods often struggle with effectively extracting both local and global image features, leading to information loss.

Purpose of the Study:

• To propose a novel dual-branch fusion network for infrared and visible image fusion.
• To enhance the feature extraction capabilities for generating fused images with rich details and contrast.

Main Methods:

• A dual-branch network combining Convolutional Neural Network (CNN) and Transformer architectures.
• Utilizing a CNN-based local feature extraction module with residual gradient and coordinate attention mechanisms.
• Employing a Transformer-based global feature extraction module to capture contextual information.

Main Results:

• The proposed network effectively extracts both local and global features from infrared and visible images.
• Experimental results on various datasets demonstrate superior performance over existing advanced fusion algorithms.
• Fused images exhibit improved target information, details, and contrast.

Conclusions:

• The dual-branch fusion network significantly advances infrared and visible image fusion.
• Integrating CNN and Transformer provides a powerful approach for comprehensive feature extraction.
• The method offers a promising solution for applications requiring high-quality fused images.