Multimodal deep learning model for predicting microsatellite instability in colorectal cancer by contrast-enhanced
Kai Tang1, Ruiling She2, Guangyuan Chen2
1Department of Radiology, The Third Affiliated Hospital, Southern Medical University (Academy of Orthopedics, Guangdong Province), 183 Zhongshan Da Dao Xi, Guangzhou, Guangdong 510630, China; School of Biomedical Engineering, Southern Medical University, Guangzhou, China; Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou, Guangdong, China; Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology, Southern Medical University, Guangzhou, Guangdong, China.
Objectives:
To develop and validate a multimodal deep learning (DL) model that integrates preoperative contrast-enhanced computed tomography (CECT) and postoperative whole-slide images (WSIs) to predict microsatellite instability (MSI) status in colorectal cancer (CRC).
Materials And Methods:
This retrospective, multicenter study enrolled 305 CRC patients with paired CECT and WSIs. Patients from Center I and II were allocated to the training (n = 169) and internal validation (n = 85) sets, while those from Center III formed the external test set (n = 51). Pathology-based DL (PathDL) and venous-phase CECT (VPDL) models were constructed using EfficientNet-b0 and ResNet 101 architectures, respectively. A fusion model (F-VP-PathDL, Fusion of venous phase CT and pathology with deep learning) was developed using an adaptive residual network to integrate features from both modalities. Model performance was evaluated using area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and F1 score.
Results:
The F-VP-PathDL model achieved strong performance on the internal validation set, with an AUC of 0.883 (95 % CI: 0.732-0.967). On the external test set, the model achieved an AUC of 0.905 (95 % CI: 0.831-0.945), outperforming single-modality and alternative fusion models (PathDL: 0.794; VPDL: 0.858; APDL: 0.802; F-AVPDL: 0.813). The model also demonstrated robust accuracy (84.2 %, 95 % CI: 69.1 %-92.8 %), sensitivity (80.3 %, 95 % CI: 28.4 %-98.7 %), specificity (83.7 %, 95% CI: 68.8 %-93.9 %) and F1 score (0.837, 95 % CI: 0.326-0.999) on the external test set.
Conclusions:
The F-VP-PathDL model demonstrates robust generalizability across centers and offers a clinically scalable tool for MSI prediction in CRC, supporting patient stratification and informing immunotherapy decisions.
