MVFusion: Generative Representation Learning With Masked Variational Autoencoders for Multi-Modality Image Fusion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Image Processing

Background:

• Multi-modality image fusion aims to create representative images from diverse sources.
• Existing methods struggle with image degradation and extracting shared/specific information.
• Limitations exist in current frameworks' generative and representation capabilities.

Purpose of the Study:

• To propose a novel framework, MVFusion, for robust multi-modality image fusion.
• To address challenges in handling varying image quality and dataset composition.
• To enhance both generative training and representation learning in a unified model.

Main Methods:

• Developed MVFusion, a self-supervised masked variational autoencoder framework.
• Employed a self-supervised masked autoencoder to mitigate redundancy and degradation.
• Incorporated variational feature learning to preserve distinctive modality features.

Main Results:

• MVFusion demonstrates promising results in classical fusion tasks.
• Achieved effective fusion for infrared-visible, multi-focus, multi-exposure, and medical images.
• The unified framework successfully handles varying image quality and dataset compositions.

Conclusions:

• MVFusion offers a robust solution for multi-modality image fusion.
• The framework effectively addresses limitations of existing unified methods.
• MVFusion shows broad applicability across diverse image fusion domains.