Explainable machine learning model for predicting acute pancreatitis mortality in the intensive care unit

Current prediction models are suboptimal for determining mortality risk in patients with acute pancreatitis (AP); this might be improved by using a machine learning (ML) model. In this study, we aimed to construct an explainable ML model to calculate the risk of mortality in patients with AP admitted in intensive care unit (ICU) and compared it with existing scoring systems.

A gradient-boosting ML (XGBoost) model was developed and externally validated based on two public databases: Medical Information Mart for Intensive Care (MIMIC, training cohort) and the eICU Collaborative Research Database (eICU-CRD, validation cohort). We compared the performance of the XGBoost model with validated clinical risk scoring systems (the APACHE IV, SOFA, and Bedside Index for Severity in Acute Pancreatitis [BISAP]) by area under receiver operating characteristic curve (AUC) analysis. SHAP (SHapley Additive exPlanations) method was applied to provide the explanation behind the prediction outcome.

The XGBoost model performed better than the clinical scoring systems in correctly predicting mortality risk of AP patients, achieving an AUC of 0.89 (95% CI: 0.84-0.94). When set the sensitivity at 100% for death prediction, the model had a specificity of 38%, much higher than the APACHE IV, SOFA and BISAP score, which had a specificity of 1%, 16% and 1% respectively.

This model might increase identification of very low-risk patients who can be safely monitored in a general ward for management. By making the model explainable, physicians would be able to better understand the reasoning behind the prediction.