Multi-Modal MRI Image Synthesis via GAN With Multi-Scale Gate Mergence

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multi-modal magnetic resonance imaging (MRI) is vital for clinical diagnosis, with each modality offering unique anatomical insights.
• Lost or corrupted MRI sequences due to cost or time constraints hinder accurate diagnosis.
• Existing multi-modal image synthesis methods struggle with effective modality fusion.

Purpose of the Study:

• To develop an advanced generative adversarial network (GAN) model for synthesizing one MRI modality from others.
• To enhance the fusion of complementary information from different MRI modalities.
• To address the challenge of incomplete or corrupted MRI data in clinical settings.

Main Methods:

• Proposed a multi-scale gate mergence based generative adversarial network (MGM-GAN).
• Utilized multiple down-sampling branches to extract unique features from each input MRI modality.
• Introduced a gate mergence (GM) mechanism for adaptive, location-specific modality weighting.
• Integrated multi-scale feature maps using the GM module.
• Employed adversarial loss, pixel-wise loss, and gradient difference loss (GDL) for network training.

Main Results:

• MGM-GAN effectively synthesizes missing MRI modalities by intelligently fusing multi-modal data.
• The gate mergence mechanism enhances task-related information and suppresses irrelevant details.
• Experimental results show superior performance compared to state-of-the-art multi-modal image synthesis methods.
• The proposed approach successfully addresses limitations of generic fusion techniques like averaging or maximizing.

Conclusions:

• MGM-GAN offers a robust solution for synthesizing missing MRI modalities, improving diagnostic capabilities.
• The novel gate mergence mechanism represents a significant advancement in multi-modal fusion for medical imaging.
• This method has the potential to mitigate the impact of incomplete MRI datasets in clinical practice.