Mask-DiFuser: A Masked Diffusion Model for Unified Unsupervised Image Fusion

Area of Science:

• Computer Vision and Image Processing
• Artificial Intelligence and Machine Learning

Background:

• The lack of ground truth (GT) in image fusion tasks hinders model development and evaluation.
• Current fusion methods often rely on subjective, hand-crafted rules and complex loss functions.
• Existing approaches struggle with adaptability in complex, real-world scenarios.

Purpose of the Study:

• To introduce Mask-DiFuser, a novel paradigm for unsupervised image fusion.
• To overcome the challenges posed by the absence of ground truth in fusion tasks.
• To improve the aggregation of complementary contexts in fused images.

Main Methods:

• Transformed unsupervised image fusion into a dual masked image reconstruction task.
• Incorporated masked image modeling with a diffusion model.
• Employed a dual masking scheme, content encoder with attention, and semantic encoder for feature extraction and integration.

Main Results:

• Mask-DiFuser effectively aggregates complementary contexts by restoring source images from masked inputs.
• The model generates fused images by iteratively denoising a Gaussian distribution conditioned on multi-source images.
• Demonstrated superior performance over state-of-the-art (SOTA) methods across infrared-visible, medical, multi-exposure, and multi-focus fusion tasks.

Conclusions:

• Mask-DiFuser provides a robust solution for unsupervised image fusion without ground truth.
• The diffusion model, guided by natural image priors, ensures perceptually aligned fusion results.
• The proposed method significantly advances the field of image fusion across diverse applications.