Multimodal AI approach combining deep learning imaging and clinical machine learning for pancreatic cancer detection

Pancreatic ductal adenocarcinoma (PDAC) has poor prognosis due to late diagnosis, limitations of computed tomography (CT) imaging, and low accuracy of clinical biomarkers. This study aimed to develop and validate a multimodal artificial intelligence (AI) approach integrating imaging-based deep learning (DL) and clinical data-driven machine learning (ML) to improve PDAC diagnosis. A retrospective cohort of 158 patients (123 PDAC, 35 benign) undergoing pancreatic surgery was analyzed. A YOLOv8-based DL model was trained on contrast-enhanced CT scans to detect pancreatic lesions, while clinical data (age, sex, serum CA19-9) were analyzed with a Random Forest ML classifier. Predictions from both models were combined into a multimodal fusion model, optimized to maximize diagnostic accuracy. Performance metrics included precision, recall, accuracy, F1-score, and ROC-AUC. The imaging-based DL model achieved strong tumor detection performance (mAP: 87.0%, precision: 86.5%, recall: 81.2%). The clinical ML model showed excellent specificity (precision: 100%, ROC-AUC: 0.931) but limited sensitivity (60%). The multimodal AI fusion model outperformed both individual models, significantly improving sensitivity, specificity, and overall diagnostic accuracy. A multimodal AI strategy integrating DL imaging analysis with ML-based clinical predictions markedly enhances diagnostic performance in pancreatic cancer. This approach offers potential as an effective decision-support tool, facilitating earlier diagnosis and optimized clinical decision-making.