Related Experiment Videos
Development and external validation of an explainable machine learning model for in-hospital mortality risk
Liusheng Xu1, Wei Cao2, Zefan Huang3
1Department of Clinical Laboratory, The Second People's Hospital of Changzhou, the Third Affiliated Hospital of Nanjing Medical University, Changzhou, Jiangsu, China.
Background:
Early risk stratification for in-hospital mortality remains challenging in intensive care unit (ICU) patients with heart failure (HF) because of substantial clinical heterogeneity and complex pathophysiological interactions. Explainable machine learning (ML) may offer a practical and transparent approach to prognostic assessment in this high-risk population.
Methods:
This retrospective dual-cohort study included 18,526 ICU patients with HF from the MIMIC-IV database (2008-2022) as the development cohort and 314 consecutive ICU patients with HF from an independent hospital-based cohort from August 1, 2023, to May 31, 2026, as the external validation cohort. Candidate predictors available within 24 h of ICU admission were screened using the Boruta algorithm and least absolute shrinkage and selection operator regression. Five ML algorithms were developed in the development cohort and compared using 5-fold nested cross-validation for internal validation. Model interpretability was assessed using Shapley Additive Explanations (SHAP).
Results:
Nine routinely available predictors were retained for final model development. Among the candidate algorithms, Extreme Gradient Boosting (XGBoost) showed the most balanced overall performance and was selected as the final model. In internal validation using nested cross-validation, the XGBoost model achieved the highest validation area under the receiver operating characteristic curve (AUC), at 0.704 (95% CI 0.682-0.726). In the external validation cohort, the final model achieved an AUC of 0.877 (95% CI 0.820-0.933) and a Brier score of 0.079 (95% CI 0.061-0.099). However, this apparently stronger discrimination should be interpreted cautiously in light of cohort differences and the limited number of external events. SHAP analysis identified blood urea nitrogen, age, and white blood cell count as the most influential predictors.
Conclusion:
An explainable XGBoost-based model using nine routinely available early clinical variables showed promising performance for predicting in-hospital all-cause mortality in ICU patients with HF. This interpretable tool may support early risk stratification and individualized clinical management in critically ill patients with HF. Further large multicenter external validation is warranted.