Self-Supervised Multi-Scale Multi-Modal Graph Pool Transformer for Sellar Region Tumor Diagnosis

Area of Science:

• Neuroimaging
• Artificial Intelligence in Medicine
• Oncology

Background:

• Sellar region tumors impact the central nervous system, necessitating early and accurate diagnosis for effective patient treatment and recovery.
• Magnetic resonance imaging (MRI) is crucial for detecting sellar region tumors, but challenges persist due to limited and imbalanced datasets.
• Current diagnostic methods struggle with the complexities of multi-modal MRI data for sellar region tumors.

Purpose of the Study:

• To propose a novel self-supervised multi-scale multi-modal graph pool Transformer (MMGPT) network for enhanced classification of sellar region tumors.
• To improve the fusion of small and imbalanced multi-modal MRI data for sellar region tumors.
• To enhance the robustness and accuracy of sellar region tumor diagnosis through advanced deep learning techniques.

Main Methods:

• Development of a novel self-supervised multi-scale multi-modal graph pool Transformer (MMGPT) network.
• Implementation of a contrastive learning equipped auto-encoder (CAE) for self-supervised learning (SSL) to extract detailed inter-sample information.
• Utilizing a hybrid loss function to mitigate performance degradation caused by data imbalance.

Main Results:

• The proposed MMGPT network demonstrated superior performance compared to state-of-the-art methods in sellar region tumor classification.
• The model achieved higher accuracy and Area Under the Curve (AUC) in classifying sellar region tumors.
• Enhanced feature interaction between multi-modal MRI data led to a more robust diagnostic model.

Conclusions:

• The MMGPT network effectively addresses the challenges of small and imbalanced datasets in sellar region tumor diagnosis.
• Self-supervised learning and multi-modal fusion are key to improving diagnostic accuracy for sellar region tumors.
• The proposed method offers a promising advancement for the early and accurate detection of sellar region tumors using MRI.