Multi-Modal Modality-Masked Diffusion Network for Brain MRI Synthesis With Random Modality Missing.

Area of Science:

• Medical imaging
• Artificial intelligence
• Computer vision

Background:

• Medical image synthesis aims to generate missing modalities from available ones for improved diagnosis.
• Current methods often rely on cross-modal translation, limiting flexibility with varying missing modalities.
• Existing approaches struggle to effectively map across multiple modalities and handle diverse missing data scenarios.

Purpose of the Study:

• To develop a unified network for flexible multi-modal medical image synthesis.
• To address limitations of existing cross-modal translation methods in handling arbitrary missing modalities.
• To improve the construction of a common latent space and enhance model representation ability.

Main Methods:

• Proposes the Multi-modal Modality-masked Diffusion Network (M2DN) using a "progressive whole-modality inpainting" approach.
• Treats missing modalities as noise and synthesizes them alongside self-reconstruction of available modalities.
• Introduces a modality-mask scheme as a condition for the diffusion model to encode modality availability.

Main Results:

• M2DN demonstrates superior performance in synthesizing missing medical imaging modalities compared to state-of-the-art methods.
• Achieved significant improvements in downstream segmentation tasks using synthesized images.
• Showcased excellent generalizability across different arbitrary missing modality scenarios.

Conclusions:

• The M2DN provides a flexible and effective solution for multi-modal medical image synthesis, overcoming limitations of prior methods.
• The "progressive whole-modality inpainting" strategy and modality-masking enhance synthesis performance and model generalizability.
• This approach facilitates multi-modal diagnosis and treatment planning by enabling robust synthesis of missing imaging data.