Related Experiment Video
Updated: May 17, 2026
A Cognitive Fusion-guided Prostate Biopsy Using Multiparametric Magnetic Resonance Imaging and Transrectal Ultrasound
Published on: March 21, 2025
MRI- and report-based multimodal model with SHAP-based explanation for preoperative prediction of deep stromal
Raoying Xie1, Yao Ai1, Anqi Bao1
1Department of Radiation and Medical Oncology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China.
Objectives:
Depth of stromal invasion (DSI) is a key prognostic factor significantly influencing treatment decisions in early-stage cervical cancer (ESCC). This study aims to develop an explainable multimodal data fusion model integrating MRI, radiology reports, and clinical variables for the preoperative assessment of DSI risk.
Materials And Methods:
Radiomic features were extracted from preoperative sagittal T2-weighted imaging (T2WI). Bidirectional Encoder Representations from Transformers (BERT) features were derived from the corresponding radiology reports using natural language processing (NLP). Key BERT and radiomic features were selected using the least absolute shrinkage and selection operator (LASSO). Independent clinical risk factors were identified through univariate analysis, followed by multivariate logistic regression. Five machine learning algorithms were employed to construct the clinical (C), text (T), radiomic (R), and multimodal fusion models (T + R, T + R + C). The Shapley Additive exPlanation (SHAP) method interpreted the optimal fusion model.
Results:
Overall, 498 radiomic features (from T2WI) and 384 BERT-derived features (from reports) were extracted per patient. LASSO selected eleven BERT features and nine radiomic features; multivariate analysis identified two independent clinical risk factors. The text-radiomic-clinical fusion model (T + R + C) outperformed all other models, achieving AUCs of 0.912, 0.874, and 0.890 in the training, internal validation, and external validation cohorts, respectively. SHAP analysis revealed that eight BERT features, five radiomic features, and two clinical features ranked among the top 15 most influential predictors.
Conclusion:
The explainable multimodal model improves preoperative DSI risk evaluation in ESCC. However, given the small external cohort (n = 20), these results are preliminary and require further independent multi-center validation before clinical application.
Critical Relevance:
The developed model significantly improves preoperative DSI prediction in ESCC through noninvasive imaging analysis, thereby refining risk assessment and guiding precision oncology interventions.
Key Points:
DSI is a key prognostic factor significantly influencing treatment decisions in ESCC. The explainable multimodal fusion model was established. The accuracy of preoperative DSI risk evaluation was beneficial for clinical decision-making.