Related Experiment Video
Updated: Aug 8, 2026
Advanced Animal Model of Colorectal Metastasis in Liver: Imaging Techniques and Properties of Metastatic Clones
Published on: November 30, 2016
A CLIP-Based Multimodal Endorectal Ultrasound Model for Predicting Metachronous Liver Metastases in Rectal Cancer
Hang Yi1, Min Liu2, Hanchen Zhang1
1Department of Medical Ultrasonics, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China (H.Y., H.Z., L.C., J.C., X.L., M.S., G.L.); Guangdong Provincial Key Laboratory of Colorectal and Pelvic Floor Diseases, The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China (H.Y., H.Z., L.C., J.C., X.L., M.S., G.L.); Biomedical Innovation Center, The Sixth Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China (H.Y., H.Z., L.C., J.C., X.L., M.S., G.L.).
A new multimodal model using contrastive language-image pretraining (CLIP) and 3D endorectal ultrasound (3D-ERUS) shows promise for predicting liver metastasis in rectal cancer patients. Integrating imaging and reports, with clinicopathological fusion, improves risk stratification.
Area of Science:
- Medical Imaging
- Artificial Intelligence in Oncology
- Oncology
Background:
- Metachronous liver metastasis (MLM) is a significant concern for rectal cancer patients post-surgery.
- Accurate prediction of MLM is crucial for personalized treatment strategies.
- Current prediction models may not fully leverage multimodal data.
Purpose of the Study:
- To develop and validate a contrastive language-image pretraining (CLIP)-based multimodal model for predicting MLM.
- To integrate pretreatment multiplanar 3D endorectal ultrasound (3D-ERUS) images and ERUS reports.
- To assess the performance of a clinicopathological fusion model for enhanced risk stratification.
Main Methods:
- A two-center retrospective study included 394 rectal cancer patients with pretreatment 3D-ERUS.
- Chinese-CLIP encoded multiplanar 3D-ERUS images and reports.
- Balanced logistic regression and clinicopathological fusion models were developed and validated.
Main Results:
- The multimodal model achieved AUCs ranging from 0.743 to 0.824 across cohorts.
- The fusion model demonstrated superior performance with AUCs from 0.808 to 0.850.
- The fusion model significantly improved discrimination (IDI) and reclassification (NRI) compared to clinical and multimodal models.
Conclusions:
- CLIP-based integration of 3D-ERUS images and reports shows potential for MLM prediction.
- Clinicopathological fusion further enhances risk stratification accuracy.
- Larger external validation is warranted to confirm these findings.