Mask-aware foundational-model embeddings for 18F-FDG-PET/CT prognosis in multiple myeloma.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in oncology
• Radiomics and deep learning

Background:

• Predicting progression-free survival (PFS) in multiple myeloma (MM) is crucial for treatment planning.
• Current prognostic models often rely on clinical data or radiomics, with potential for improvement using advanced imaging features.
• Foundational segmentation models offer novel ways to extract information from medical images.

Purpose of the Study:

• To evaluate the efficacy of internal memory states from a medical segmentation model (MedSAM2) as compact, mask-aware embeddings for MM PFS prediction.
• To assess the impact of late fusion of PET, CT, and clinical data on prognostic performance.
• To determine if these embeddings can serve as data-efficient imaging biomarkers.

Main Methods:

• Analysis of 227 newly diagnosed MM patients with whole-body [18F]FDG PET/CT and clinical data.
• Extraction of spatio-temporal memory tensors from MedSAM2 using mask-derived bounding boxes for spine-dilated and full skeleton regions.
• Comparison of channel×memory averaging and depth-attention pooling for per-study embedding generation, followed by late fusion with clinical data and evaluation using DeepSurv.

Main Results:

• Image-only models using averaging achieved a c-index of 0.659±0.015 (PET, spine-dilated), comparable to radiomics.
• Multimodal models (PET/CT + clinical data) improved discrimination to 0.710±0.032 (CT, spine-dilated), outperforming clinical-only baselines by ~6.5%.
• Averaging downsampling strategy consistently outperformed depth-attention, and PET embeddings showed better performance than CT in image-only settings.

Conclusions:

• Mask-aware memory embeddings from foundational segmentation models are effective imaging biomarkers for MM PFS prediction.
• Fusion with clinical data significantly enhances risk stratification compared to clinical-only or radiomics approaches.
• This method provides a practical, data-efficient strategy for prognostic modeling in small medical cohorts without manual feature engineering.