Development of a machine learning-based early prediction model for disease severity in acute pancreatitis

Severe acute pancreatitis (SAP) represents the most severe form of acute pancreatitis (AP) and is associated with considerable morbidity, mortality, and socioeconomic burden. Early identification of critically ill patients is crucial for disease management in AP. However, commonly used scoring systems are often complex, time-consuming, and lack interpretability. Therefore, the purpose of this study is to develop a machine learning-based model for early prediction of severity in AP and to perform interpretability analysis.

A total of 528 AP patients meeting the inclusion criteria were screened via the hospital electronic medical record system and randomly divided into training and testing sets at a 7:3 ratio. Least Absolute Shrinkage and Selection Operator (LASSO) regression was employed for variable selection. Four machine learning algorithms, Logistic Regression, extreme Gradient Boosting (XGBoost), Random Forest, and Support Vector Machine (SVM), were utilized to construct prediction models. The optimal model was identified by comparing the following metrics between the training and testing sets: area under the receiver operating characteristic curve (AUC), Kappa value, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score. In addition, the calibration curve and Decision Curve Analysis (DCA) within the testing set were evaluated. SHapley Additive exPlanations (SHAP) analysis was applied to interpret the selected model.

Comparative evaluation of the machine learning models revealed that the SVM model demonstrated the best overall performance. The SVM model achieved an AUC = 0.819, Kappa = 0.509, accuracy = 0.861, sensitivity = 0.615, specificity = 0.909, PPV = 0.571, NPV = 0.923, and F1 score = 0.593 on the testing set, respectively. SHAP analysis indicated that AP severity increased with higher SHAP values. Elevated levels of the Simple Prognostic Score (SPS), respiratory rate, and High density lipoprotein cholesterol (HDL-C), along with the presence of peritoneal effusion, were associated with the occurrence of SAP.

The developed SVM model shows promising potential for distinguishing AP severity levels. It holds significant clinical utility, particularly for its high effectiveness in excluding non-SAP patients.