Related Experiment Video
Updated: Oct 3, 2025
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Dynamic Mortality Risk Predictions for Children in ICUs: Development and Validation of Machine Learning Models
Eduardo A Trujillo Rivera1, James M Chamberlain2, Anita K Patel3
1George Washington University School of Medicine and Health Sciences, Washington, DC.
Insights
A machine learning model accurately tracks hospital mortality risk in pediatric intensive care units by analyzing physiology and care intensity over time. This Criticality Index-Mortality (CI-M) tool aids in real-time patient monitoring.
Area of Science:
- Pediatric critical care medicine
- Machine learning in healthcare
- Health informatics
Background:
- Accurate, real-time assessment of mortality risk is crucial for pediatric intensive care unit (PICU) patients.
- Existing methods may not adequately capture dynamic changes in patient status during ICU stays.
Purpose of the Study:
- To assess a machine learning method for serially updated mortality risk estimation in critically ill children.
- To develop and validate a model that dynamically tracks mortality risk throughout an ICU admission.
Main Methods:
- Retrospective analysis of a national database (Health Facts) including 27,354 pediatric ICU admissions (2009-2018).
- Development of the Criticality Index-Mortality (CI-M) model using physiology, therapy, and care intensity data, updated every 6 hours up to 180 hours.
- Model performance evaluated using discrimination (area under the ROC curve) and calibration (Hosmer-Lemeshow test).
Main Results:
- The CI-M model demonstrated strong discrimination with an overall AUC of 0.852.
- The model showed good calibration across 29 time periods, indicating reliable risk estimates.
- Clinical validity was confirmed through analysis of patient trajectories and greater volatility observed in non-survivors.
Conclusions:
- Machine learning models integrating patient physiology, therapy, and care intensity can effectively monitor dynamic changes in hospital mortality risk in the ICU.
- The CI-M framework offers a promising approach for real-time monitoring of clinical improvement and deterioration in critically ill children.
Objectives:
Assess a machine learning method of serially updated mortality risk.
Design:
Retrospective analysis of a national database (Health Facts; Cerner Corporation, Kansas City, MO).
Setting:
Hospitals caring for children in ICUs.
Patients:
A total of 27,354 admissions cared for in ICUs from 2009 to 2018.
Interventions:
None.
Main Outcome:
Hospital mortality risk estimates determined at 6-hour time periods during care in the ICU. Models were truncated at 180 hours due to decreased sample size secondary to discharges and deaths.
Measurements And Main Results:
The Criticality Index, based on physiology, therapy, and care intensity, was computed for each admission for each time period and calibrated to hospital mortality risk (Criticality Index-Mortality [CI-M]) at each of 29 time periods (initial assessment: 6 hr; last assessment: 180 hr). Performance metrics and clinical validity were determined from the held-out test sample (n = 3,453, 13%). Discrimination assessed with the area under the receiver operating characteristic curve was 0.852 (95% CI, 0.843-0.861) overall and greater than or equal to 0.80 for all individual time periods. Calibration assessed by the Hosmer-Lemeshow goodness-of-fit test showed good fit overall (p = 0.196) and was statistically not significant for 28 of the 29 time periods. Calibration plots for all models revealed the intercept ranged from--0.002 to 0.009, the slope ranged from 0.867 to 1.415, and the R2 ranged from 0.862 to 0.989. Clinical validity assessed using population trajectories and changes in the risk status of admissions (clinical volatility) revealed clinical trajectories consistent with clinical expectations and greater clinical volatility in deaths than survivors (p < 0.001).
Conclusions:
Machine learning models incorporating physiology, therapy, and care intensity can track changes in hospital mortality risk during intensive care. The CI-M's framework and modeling method are potentially applicable to monitoring clinical improvement and deterioration in real time.
Related Concept Videos
01:18Steps in Outbreak Investigation
01:20Actuarial Approach
Consider the example of a high-risk surgical procedure with significant early-stage mortality. A two-year clinical study is conducted,...
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
01:03Prediction Intervals
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
01:24Kaplan-Meier Approach