Learning Unified Hyper-Network for Multi-Modal MR Image Synthesis and Tumor Segmentation With Missing Modalities.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neuroscience

Background:

• Accurate brain tumor segmentation is crucial for clinical assessment and treatment planning.
• Multi-modal Magnetic Resonance Imaging (MRI) provides complementary information but may have missing modalities in real-world scenarios.
• Existing methods for handling missing modalities are computationally burdensome or yield suboptimal performance.

Purpose of the Study:

• To propose a unified and adaptive multi-modal MR image synthesis method.
• To apply this synthesis method to improve brain tumor segmentation in cases with missing modalities.
• To develop an adaptive approach for utilizing both real and synthetic MRI data.

Main Methods:

• Decomposition of multi-modal MR images into common and modality-specific features.
• Utilized a shared hyper-encoder, a graph-attention-based fusion block, and a shared hyper-decoder for image synthesis.
• Incorporated an adversarial common feature constraint and a hypernet-based modulation module for adaptive segmentation.

Main Results:

• The proposed method successfully synthesizes reasonable multi-modal MR images.
• Achieved state-of-the-art performance in brain tumor segmentation with missing modalities.
• Demonstrated the effectiveness of feature-level and image-level completion for segmentation.

Conclusions:

• The unified and adaptive synthesis method effectively addresses the challenge of missing MRI modalities.
• The approach enhances the accuracy and robustness of brain tumor segmentation.
• This work offers a promising solution for clinical applications requiring complete multi-modal MRI data.