Related Experiment Video
Updated: Jul 9, 2025
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
Development and Temporal Validation of a Machine Learning Model to Predict Clinical Deterioration
Henry P Foote1, Zohaib Shaikh2,3, Daniel Witt2,4
1Divisions of Pediatric Cardiology.
Insights
A new machine learning model using electronic health records can predict clinical deterioration in pediatric patients earlier than current methods. This advanced tool shows promise for improving early detection of critical events in hospitalized children.
Area of Science:
- Pediatric critical care medicine
- Health informatics
- Machine learning in healthcare
Background:
- Existing early warning scores for pediatric inpatients have variable performance and limited clinical feature utilization.
- Predicting clinical deterioration in hospitalized children is crucial for timely intervention and improved outcomes.
Purpose of the Study:
- To develop and evaluate a machine learning model using electronic health record data to predict clinical deterioration in pediatric inpatients.
- To compare the performance of the machine learning model against the institutional Pediatric Early Warning Score (I-PEWS).
Main Methods:
- A retrospective cohort of 17,630 pediatric encounters was used for model development.
- Two machine learning models, light gradient boosting machine (LGBM) and random forest, were trained on 542 features.
- Models were compared with the institutional Pediatric Early Warning Score (I-PEWS) using internal and temporal validation cohorts.
Main Results:
- The LGBM model demonstrated superior performance in predicting the composite outcome of unplanned ICU transfer or mortality, with a higher area under the receiver operating characteristic curve (AUROC) compared to I-PEWS (0.785 vs 0.708 in temporal validation).
- The machine learning model provided a significantly longer lead time for detecting deterioration events (median 11 hours vs 3 hours).
- However, the LGBM model had a lower positive predictive value (6% vs 29%) and a higher number needed to evaluate (17 vs 3) in the temporal validation cohort.
Conclusions:
- An electronic health record-based machine learning model shows improved accuracy (AUROC) and earlier detection (lead-time) of clinical deterioration in pediatric inpatients compared to I-PEWS.
- The model can predict critical events 24 to 48 hours in advance.
- Further research is required to enhance the model's positive predictive value for successful clinical integration.
Objectives:
Early warning scores detecting clinical deterioration in pediatric inpatients have wide-ranging performance and use a limited number of clinical features. This study developed a machine learning model leveraging multiple static and dynamic clinical features from the electronic health record to predict the composite outcome of unplanned transfer to the ICU within 24 hours and inpatient mortality within 48 hours in hospitalized children.
Methods:
Using a retrospective development cohort of 17 630 encounters across 10 388 patients, 2 machine learning models (light gradient boosting machine [LGBM] and random forest) were trained on 542 features and compared with our institutional Pediatric Early Warning Score (I-PEWS).
Results:
The LGBM model significantly outperformed I-PEWS based on receiver operating characteristic curve (AUROC) for the composite outcome of ICU transfer or mortality for both internal validation and temporal validation cohorts (AUROC 0.785 95% confidence interval [0.780-0.791] vs 0.708 [0.701-0.715] for temporal validation) as well as lead-time before deterioration events (median 11 hours vs 3 hours; P = .004). However, LGBM performance as evaluated by precision recall curve was lesser in the temporal validation cohort with associated decreased positive predictive value (6% vs 29%) and increased number needed to evaluate (17 vs 3) compared with I-PEWS.
Conclusions:
Our electronic health record based machine learning model demonstrated improved AUROC and lead-time in predicting clinical deterioration in pediatric inpatients 24 to 48 hours in advance compared with I-PEWS. Further work is needed to optimize model positive predictive value to allow for integration into clinical practice.
