TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Image fusion combines infrared and visible images to create a unified representation with complementary features.
• Conventional deep learning fusion methods often overlook the importance of target objects for downstream tasks.
• Existing approaches may not effectively bridge the performance gap between pixel-level fusion and object detection.

Purpose of the Study:

• To introduce TDDFusion, a Target-Driven Dual-Branch Fusion Network, designed to enhance target object prominence in fused images.
• To address the limitations of current fusion techniques by prioritizing task-oriented optimization for object detection.
• To improve the relevance of fused images for subsequent decision-making processes.

Main Methods:

• Employs a parallel, dual-branch feature extraction network with a Global Semantic Transformer (GST) and Local Texture Encoder (LTE).
• Integrates an object detection submodule during training to backpropagate semantic loss for task-oriented fusion.
• Utilizes a novel loss function that incorporates target positional information to enhance object-specific contrast and detail.

Main Results:

• TDDFusion demonstrates superiority in preserving both global environmental information and local details compared to state-of-the-art methods.
• The model effectively balances pixel intensity and maintains the texture of target objects.
• Significant advantages were observed in downstream object detection tasks, outperforming existing fusion techniques.

Conclusions:

• TDDFusion effectively enhances target object visibility in fused infrared and visible images.
• The proposed network architecture and loss function enable task-oriented fusion, improving performance in object detection.
• This approach offers a superior solution for image fusion applications requiring accurate identification and analysis of target objects.