VDMUFusion: A Versatile Diffusion Model-Based Unsupervised Framework for Image Fusion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Image fusion integrates information from multiple source images to enhance perception and analysis.
• Diffusion models excel at generative tasks but require ground truth data, posing challenges for unsupervised image fusion.
• Existing unsupervised fusion methods struggle with direct application of diffusion models due to the lack of ground truth data.

Purpose of the Study:

• To propose a versatile diffusion model-based unsupervised framework for image fusion (VDMUFusion).
• To address the limitation of lacking ground truth data in unsupervised image fusion tasks.
• To improve the performance and quality of fused images in various unsupervised fusion applications.

Main Methods:

• Formulating image fusion as a weighted average process integrated into the diffusion sampling process.
• Developing a multi-task learning framework for simultaneous prediction of noise and fusion weights, replacing the original noise prediction network.
• Employing joint training across diverse fusion tasks to enhance noise prediction accuracy and overall performance.

Main Results:

• VDMUFusion demonstrates competitive performance across various unsupervised image fusion tasks.
• The multi-task learning framework and joint training improve noise prediction accuracy.
• The proposed method yields higher quality fused images compared to single-task training.

Conclusions:

• VDMUFusion offers a versatile and effective solution for unsupervised image fusion using diffusion models.
• The framework successfully overcomes the ground truth data limitation inherent in applying diffusion models to unsupervised tasks.
• Joint training across multiple fusion tasks significantly boosts performance and fused image quality.