A Novel Framework for Multimodal Brain Tumor Detection With Scarce Labels

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Deep Learning

Background:

• Multimodal data (MRI, CT) offers diagnostic advantages for brain tumor detection but faces challenges in information fusion.
• Existing deep learning models often rely on single modalities, and multimodal fusion methods can lead to critical information loss.
• Medical image analysis requires extensive annotated data, a resource-intensive process demanding expert time.

Purpose of the Study:

• To develop an effective unsupervised learning framework for brain tumor detection using multimodal MRI and CT data.
• To overcome the limitations of single-modality reliance and information loss during multimodal fusion.
• To address the challenge of limited annotated data in medical image analysis.

Main Methods:

• Introduced Double-SimCLR, an unsupervised learning framework based on contrastive learning with a dual-branch structure for simultaneous MRI and CT processing.
• Incorporated adaptive weight masking technology to enhance feature extraction from CT images, addressing their weak feature characteristics.
• Implemented a multimodal attention mechanism to focus on salient information, improving detection precision and robustness.

Main Results:

• Double-SimCLR achieved 93.458% accuracy, 92.463% precision, and 93.058% F1-score without substantial labeled data.
• The framework outperformed state-of-the-art (SOTA) models by 2.871% in accuracy, 2.643% in precision, and 3.098% in F1-score.
• Demonstrated the efficacy of unsupervised multimodal fusion for brain tumor detection.

Conclusions:

• Double-SimCLR provides an effective solution for brain tumor detection using multimodal MRI and CT data, even with limited labeled datasets.
• The framework's novel approach to multimodal fusion and feature extraction enhances detection performance and robustness.
• This unsupervised method holds significant potential for advancing medical image analysis and reducing the burden of data annotation.