DTFusion: Infrared and Visible Image Fusion Based on Dense Residual PConv-ConvNeXt and Texture-Contrast Compensation

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Infrared and visible image fusion integrates complementary information from two source images to create a more informative fused image.
• Existing deep learning fusion methods often use small kernels or fixed fusion strategies, limiting feature representation and overall performance.

Purpose of the Study:

• To propose a novel end-to-end infrared and visible image fusion framework, DTFusion, to overcome limitations of current deep learning approaches.
• To enhance feature extraction capabilities and improve the quality of fused images by addressing limitations in receptive field size and feature representation.

Main Methods:

• Developed DTFusion, an end-to-end framework incorporating a residual PConv-ConvNeXt module (RPCM) for efficient feature extraction with larger receptive fields.
• Introduced a texture-contrast compensation module (TCCM) utilizing gradient residuals and an attention mechanism to preserve and enhance texture details and contrast.
• Employed dense connections in the encoder and four convolutional layers for feature reconstruction.

Main Results:

• DTFusion demonstrated superior performance compared to state-of-the-art fusion methods on public datasets.
• The proposed framework achieved better results in both subjective visual quality and objective performance metrics.
• Experimental results validate the effectiveness of RPCM and TCCM in improving feature representation and fusion quality.

Conclusions:

• DTFusion offers an advanced solution for infrared and visible image fusion, significantly improving feature representation and fusion performance.
• The novel modules (RPCM and TCCM) effectively address the limitations of small receptive fields and inadequate detail compensation in existing methods.
• The proposed framework provides a robust and high-performing approach for generating informative fused images from infrared and visible sources.