Multi-modal deep learning model for predicting recurrence of moderately severe and severe acute pancreatitis

To overcome the limitations of single-modality predictors by developing and validating a multimodal model (APNet) that integrates clinical factors and contrast-enhanced CT features to predict recurrence of moderate-to-severe acute pancreatitis (MSAP/SAP).

We retrospectively collected clinical data and enhanced CT images from a total of 235 patients with moderate-to-severe AP. To rigorously evaluate model generalizability, the dataset was divided into two distinct cohorts: a Development Cohort (N = 184) for model training and internal cross-validation, and an Independent Validation Cohort (N = 51) for performance evaluation. Clinical machine learning models were first developed, followed by APNet, a multimodal deep learning model integrating ResNet- and ViT-extracted CT features with clinical risk factors through multiscale fusion.

Among single-modality approaches, the LightGBM model using clinical data achieved an AUC of 0.711, while image-based deep learning with ResNet50 reached an AUC of 0.815. The proposed multimodal fusion model, APNet, showed the best predictive performance, achieving an AUC of 0.840 on the independent test set, with corresponding accuracy, precision, recall, and F1 score of 82.35%, 66.67%, 80.00%, and 72.73%. Overall, APNet consistently outperformed all single-modality models, highlighting the complementary value of combining imaging features with clinical risk factors.

APNet effectively integrates clinical and imaging data, significantly improving prediction of recurrence in MSAP/SAP patients. This multimodal tool can help identify high-risk MSAP and SAP patients early, supporting targeted interventions and better long-term outcomes.