Detection Drives an End-to-End Fusion of Infrared and Visible Images Based on Diffusion Models

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Existing infrared and visible image fusion methods often compromise detection performance or computational efficiency.
• Current approaches struggle with either independent fusion or joint training requirements.

Purpose of the Study:

• To propose a detection-driven image fusion network (DDIF) that optimizes fused images specifically for object detection tasks.
• To address the limitations of existing fusion methods by enhancing detection performance and maintaining computational efficiency.

Main Methods:

• Reformulated image fusion as an inverse problem solved via non-differentiable optimization using diffusion model priors.
• Designed a Response Guide Learning Module (RGLM) to learn modality contributions based on detection tasks.
• Established gradient relationships for end-to-end training and employed a pre-trained, frozen detection model for flexible integration.

Main Results:

• Achieved superior object detection performance compared to state-of-the-art (SOTA) methods.
• Produced high-quality fused images that preserve source modality information.
• Demonstrated flexible integration with various detection networks while maintaining computational efficiency.

Conclusions:

• The proposed DDIF method effectively optimizes image fusion for object detection.
• DDIF offers a balance between detection performance, fusion quality, and computational efficiency.
• This approach provides a flexible and effective solution for multi-modal image fusion in detection tasks.