M ² FTrans: Modality-Masked Fusion Transformer for Incomplete Multi-Modality Brain Tumor Segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Biology

Background:

• Brain tumor segmentation is crucial for diagnosis and treatment planning.
• Current methods often require complete multi-modality MRI data, which is not always available clinically.
• Existing fusion strategies struggle with missing modalities, leading to degraded segmentation performance.

Purpose of the Study:

• To propose a novel framework, M²FTrans, for robust brain tumor segmentation under incomplete multi-modality MRI settings.
• To develop an effective cross-modality feature fusion strategy that addresses the challenge of missing data.
• To improve the reliability and accuracy of brain tumor segmentation in real-world clinical scenarios.

Main Methods:

• Developed M²FTrans, a framework utilizing modality-masked fusion transformers.
• Introduced learnable fusion tokens and masked self-attention to handle missing inputs and capture long-range dependencies.
• Incorporated spatial weight attention and channel-wise fusion transformers for modality re-balancing and feature redundancy reduction.

Main Results:

• M²FTrans demonstrated superior performance in brain tumor segmentation across various incomplete multi-modality settings.
• The framework significantly outperformed state-of-the-art methods on the BraTS2018, BraTS2020, and BraTS2021 datasets.
• The proposed fusion strategy proved robust to missing modalities, maintaining high segmentation accuracy.

Conclusions:

• M²FTrans offers a robust and effective solution for brain tumor segmentation with incomplete multi-modality MRI data.
• The framework's ability to handle missing modalities represents a significant advancement for clinical applications.
• The study highlights the potential of modality-masked fusion transformers in medical image analysis.