Infrared and Visible Image Fusion via Residual Interactive Transformer and Cross-Attention Fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Deep learning-based infrared and visible image fusion methods struggle to integrate global and local features.
• This limitation leads to poor texture detail and low contrast in fused images, particularly for thermal targets.

Purpose of the Study:

• To propose an advanced infrared and visible image fusion network.
• To improve the extraction of dependencies between global and local information for enhanced fusion results.

Main Methods:

• A novel network incorporating a residual dense module for shallow feature extraction.
• Utilizing a residual interactive transformer to extract and interact global and local features.
• Employing a cross-attention fusion module for effective feature map integration and an image reconstruction network for final output.

Main Results:

• The proposed method significantly improves the clarity of visible texture details in fused images.
• Enhanced contrast and visibility of infrared thermal targets are achieved.
• Experimental results on RoadScene, TNO, and M3FD datasets demonstrate superior performance compared to nine other fusion methods.

Conclusions:

• The developed infrared and visible image fusion network effectively addresses limitations of existing methods.
• The integration of residual interactive transformer and cross-attention fusion leads to superior fusion performance, offering richer details and better contrast.