Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multicancer Cohort.

Area of Science:

• Oncology
• Medical Informatics
• Machine Learning

Background:

• Electronic health records (EHR) contain valuable structured and unstructured data for patient health assessment.
• Multimodal machine learning (MML) integrates diverse data types for a comprehensive patient view.
• Predicting cancer metastasis risk is crucial for timely and effective treatment planning.

Purpose of the Study:

• To develop and evaluate a framework for predicting cancer metastasis risk one month prior to diagnosis.
• To leverage multimodal machine learning on EHR data for enhanced predictive accuracy.
• To compare the performance of deep learning (DL) and traditional classifiers using various fusion strategies.

Main Methods:

• Analysis of four cancer cohorts (breast, colon, lung, prostate) from Karolinska University Hospital.
• Inclusion of demographics, comorbidities, laboratory results, medications, and clinical text from EHR.
• Comparison of single-modality and multimodal classifiers (traditional and DL) using TRIPOD 2a design and an 80-20 split.
• Performance evaluation via AUROC, AU-PRC, F1 score, sensitivity, and specificity; SHAP analysis for interpretability.

Main Results:

• Intermediate fusion yielded the highest F1 scores for breast (0.845), colon (0.786), and prostate cancer (0.845).
• Lung cancer showed strong performance with intermediate fusion (0.819), while a text-only model achieved the highest F1 score (0.829).
• Deep learning classifiers consistently outperformed traditional models; colon cancer cohort performance was limited by data size.

Conclusions:

• Fusion strategies offer varied strengths, with intermediate fusion generally providing optimal results.
• The choice of fusion strategy should be tailored to specific data characteristics and clinical needs.
• Multimodal machine learning, particularly with intermediate fusion, shows significant promise for predicting metastasis risk from EHR data.