Development and external evaluation of an interpretable machine-learning model for early prediction of organ failure in higher-risk acute pancreatitis patients: A multicentre cohort study

Organ failure (OF) is the most important determinant of prognosis in acute pancreatitis (AP) and its duration defines disease severity. Early identification of individuals at high risk of developing OF is crucial. We aimed to develop machine-learning models, benchmarked against a feedforward multilayer-perceptron (MLP) neural network, to predict new-onset OF at admission using static admission-day variables. In this study, data were extracted from MIMIC-IV (development cohort) and a multicentre AP cohort from two Chinese tertiary teaching hospitals (evaluation cohort), which excluded mild AP and therefore comprised patients requiring ICU-level or high-dependency care. Features were selected with Boruta and LASSO. Five machine-learning models and one multilayer-perceptron benchmark were developed in MIMIC-IV and externally evaluated in the Xiangya cohort without updating. SHapley Additive exPlanations (SHAP) were used to visualize decision-making patterns and individual prediction interpretations. The best-performing model was deployed as an interactive, web-based tool. We found that in the discovery cohort, 341 (23.9%) of 1429 AP patients developed new-onset OF within 28 days of admission while 45 of 216 patients (20.8%) in the validation cohort developed OF. Boruta and LASSO algorithms identified six key predictors including blood urea nitrogen, platelets, triglyceride-glucose index, albumin, white blood cells, and partial thromboplastin time, which were used to construct the ML and MLP models. XGBoost gave the best discrimination on external evaluation (AUC 0.837, 95% CI 0.771-0.902). Logistic recalibration improved calibration, and the recalibrated XGBoost model was implemented as a web-based research prototype (Decent app). In conclusion, XGBoost predicted new-onset OF with good discrimination in a severity-enriched AP cohort, and SHAP made individual predictions interpretable. Model selection, threshold selection and recalibration all used the evaluation cohort, so the deployed model still requires an independent cohort. Prospective usability and clinical-impact studies are needed before clinical use.