Multimodal MR Synthesis via Modality-Invariant Latent Representation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neuroscience

Background:

• Magnetic Resonance Imaging (MRI) synthesis is crucial for medical diagnosis.
• Existing MRI synthesis methods often struggle with missing data or require specific training curricula.
• Developing robust and versatile MRI synthesis models is an ongoing challenge.

Purpose of the Study:

• To develop a multi-input, multi-output fully convolutional neural network for robust MRI synthesis.
• To create a model that can handle missing input modalities and learn a shared latent space.
• To investigate the impact of incorporating segmentation masks for improved synthesis and lesion generation.

Main Methods:

• A fully convolutional neural network (FCNN) architecture was designed for end-to-end training.
• Input modalities were embedded into a shared, modality-invariant latent space.
• A learned decoder fused latent representations to generate target MRI modalities.
• Segmentation masks were integrated to enhance model performance and lesion synthesis.

Main Results:

• The model demonstrated robustness to missing data, outperforming state-of-the-art methods.
• Utilizing multiple input modalities significantly improved synthesis accuracy by leveraging a common latent space.
• Incorporating segmentation masks reduced error and enabled the generation of synthetic lesions.
• Statistically significant improvements were achieved on ISLES and BRATS datasets, including non-skull-stripped brain images.

Conclusions:

• The proposed FCNN model offers a robust and flexible approach to MRI synthesis.
• Learning a shared latent space effectively integrates information from multiple modalities.
• The model's ability to utilize segmentation masks enhances its utility for both standard synthesis and lesion generation tasks.