MD-SA2: optimizing Segment Anything 2 for multimodal, depth-aware brain tumor segmentation in sub-Saharan populations

Machine learning algorithms are emerging as valuable aides for radiologists in medical image segmentation due to their accuracy and speed. However, existing approaches, including both conventional machine learning and Segment Anything (SA)-based models, face challenges with the complex, multimodal, and varied quality of magnetic resonance imaging (MRI) scan images used for brain tumor segmentation. To address these challenges, we propose MD-SA2, adapting Segment Anything 2 (SA2) to medical image segmentation and introducing a lightweight U-Net "aggregator" model.

Various modifications are incorporated to enhance segmentation accuracy and throughput. SA2 is first customized and fine-tuned for greater efficiency than the original Segment Anything. Slices from multiple image modalities are concatenated for input into the image encoder to improve the delineation of tumor subtypes. In addition, a lightweight U-Net aggregator model is integrated with SA2 to introduce depth awareness. The 2023 BraTS-Africa dataset, containing low-resolution MRI images from 60 sub-Saharan patients, is used to evaluate the algorithm's performance.

MD-SA2 attains notable improvements over existing approaches under challenging data circumstances. It achieves a tenfold cross-validated, statistically significant improvement over current methods with a 0.7893 Dice coefficient. It also reaches a higher Intersection over Union and lower 95% Hausdorff distance metrics. An ablation study verifies the impact of key components.

MD-SA2 displays strong potential for supporting the diagnosis and treatment planning of brain tumors. It may contribute to narrowing health inequities, especially in medically underserved areas where data quantity and quality limitations reduce the efficacy of traditional automated approaches.