Computerized Classification Method for Glioma Molecular Subtypes on Brain MR Images Using SAM-Med3D with Low-Rank...

It is important to identify the molecular subtypes of gliomas to determine appropriate management strategies for patients. However, genetic testing requires tumor tissue obtained through surgical resection, which imposes a considerable burden on patients. Hence, we propose a computerized molecular subtype classification method based on brain magnetic resonance (MR) images using a pretrained 3D foundation model. Our dataset consists of T1-weighted (T1w), T2-weighted (T2w), fluid-attenuated inversion recovery (FLAIR), and contrast-enhanced T1-weighted (T1ce) brain MR images from the BraTS2020 dataset. The proposed model was trained and evaluated using this dataset, which comprises data from 148 patients for training and 70 patients for testing. Our proposed SAM-Med3D-based multi-modal network incorporates four modality-specific 3D image encoders for the T1w, T2w, FLAIR, and T1ce images. Each encoder is efficiently adapted using low-rank adaptation, and a classification head is introduced for glioma molecular subtype classification. Multi-modal MR images are independently processed by the modality-specific encoders to extract image embeddings. These embeddings, together with the prompt embeddings generated by the 3D prompt encoder, are integrated by the 3D mask decoder to produce tumor segmentation outputs. The shared encoder features are concatenated and sent to the classification head for glioma molecular subtype classification. The area under the curve for the proposed method was 0.931, exceeding that of the conventional networks such as SGPNet (0.827), MA-MTLN (0.902), and MTTU-Net (0.910). This result indicates that the proposed SAM-Med3D-based network could enable effective and accurate molecular subtype classification using multi-modal brain MR images.