Related Experiment Video
Updated: Jul 8, 2026
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Development and Internal-External Validation of a Post-Operative Mortality Risk Calculator for Pediatric Surgical
Lauren Eyler Dang1, Greg Klazura2, Ava Yap3
1NIAID Biostatistics Research Branch, Bethesda, MD, USA.
Insights
A new machine learning algorithm accurately predicts pediatric post-operative mortality in low-resource settings. This tool can improve clinical care and resource allocation for children undergoing surgery worldwide.
Area of Science:
- Global Health
- Medical Informatics
- Pediatric Surgery
Background:
- Pediatric surgical outcomes in low- and middle-income countries (LMICs) are critical.
- Developing accurate mortality prediction tools for these settings is essential for improving care.
Purpose of the Study:
- To develop and validate a machine learning-based mortality risk algorithm for pediatric surgery patients.
- To assess the algorithm's performance at KidsOR sites in 14 LMICs.
Main Methods:
- A SuperLearner machine learning algorithm was trained on a database of over 21,000 pediatric surgical patients.
- Data were collected retrospectively and prospectively from June 2018 to June 2023 across 20 KidsOR sites.
- Algorithm performance was evaluated using internal-external cross-validation, focusing on AUC and calibration.
Main Results:
- The best-performing algorithm was an extreme gradient boosting model with a cross-validated AUC of 0.945.
- External validation demonstrated excellent discrimination (AUC 0.864), though re-calibration may be needed for new sites.
- The overall post-operative mortality rate in the study cohort was 3.1%.
Conclusions:
- The developed KidsOR post-operative mortality risk algorithm shows outstanding predictive performance.
- The model can inform clinical practice and guide resource allocation in LMICs.
- Further validation and potential re-calibration are recommended for deployment at new sites.
Background:
The purpose of this study was to develop and validate a mortality risk algorithm for pediatric surgery patients treated at KidsOR sites in 14 low- and middle-income countries.
Methods:
A SuperLearner machine learning algorithm was trained to predict post-operative mortality by hospital discharge using the retrospectively and prospectively collected KidsOR database including patients treated at 20 KidsOR sites from June 2018 to June 2023. Algorithm performance was evaluated by internal-external cross-validated AUC and calibration.
Findings:
Of 23,905 eligible patients, 21,703 with discharge status recorded were included in the analysis, representing a post-operative mortality rate of 3.1% (671 mortality events). The candidate algorithm with the best cross-validated performance was an extreme gradient boosting model. The cross-validated AUC was 0.945 (95% CI 0.936 to 0.954) and cross-validated calibration slope and intercept were 1.01 (95% CI 0.96 to 1.06) and 0.05 (95% CI -0.10 to 0.21). For Super Learner models trained on all but one site and evaluated in the holdout site for sites with at least 25 mortality events, overall external validation AUC was 0.864 (95% CI 0.846 to 0.882) with calibration slope and intercept of 1.03 (95% CI 0.97 to 1.09) and 1.18 (95% CI 0.98 to 1.39).
Interpretation:
The KidsOR post-operative mortality risk algorithm had outstanding cross-validated discrimination and strong cross-validated calibration. Across all external validation sites, discrimination of Super Learner models trained on the remaining sites was excellent, though re-calibration may be necessary prior to use at new sites. This model has the potential to inform clinical practice and guide resource allocation at KidsOR sites world-wide.
Type Of Study And Level Of Evidence:
Observational Study, Level III.
