Related Experiment Video
Updated: Jun 22, 2026
Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale
Published on: August 25, 2014
Development, External Validation, and Biomolecular Corroboration of Interoperable Models for Identifying Critically
Christopher M Horvat1,2, Amie J Barda3, Eddie Perez Claudio4
1Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA.
Insights
Machine learning models can predict neurologic morbidity in critically ill children, aiding early detection and intervention. Biomarker correlations support these predictive capabilities for improved neurodevelopmental outcomes.
Area of Science:
- Pediatric Critical Care Medicine
- Neurocritical Care
- Machine Learning in Healthcare
Background:
- Declining mortality in pediatric critical care necessitates focus on neurodevelopmental outcomes.
- Early identification of neurologic morbidity is crucial for timely interventions and preserving neurodevelopmental trajectories.
Purpose of the Study:
- Develop and validate machine-learning models to identify acquired neurologic morbidity in critically ill children.
- Assess the correlation between model predictions and serum-based brain injury biomarkers.
Main Methods:
- Retrospective cohort study at two quaternary children's hospitals.
- Developed extreme gradient boosting (XGBoost) models using encounter data.
- Validated models externally and optimized with spline recalibration.
- Assessed correlation with glial fibrillary acidic protein (GFAP) biomarker levels.
Main Results:
- The development site model achieved an F1-score of 0.54 and AUROC of 0.82.
- The generalizable model at the validation site had an F1-score of 0.37 and AUROC of 0.81.
- Recalibrated model achieved a Brier score of 0.04.
- GFAP levels significantly correlated with model predictions (r=0.34, P=0.007).
Conclusions:
- Demonstrated well-performing machine learning models for predicting pediatric neurologic morbidity.
- Biomolecular corroboration supports the utility of these predictive models.
- Further prospective assessment of biomarker-coupled risk models in pediatric critical illness is warranted.
Importance:
Declining mortality in the field of pediatric critical care medicine has shifted practicing clinicians' attention to preserving patients' neurodevelopmental potential as a main objective. Earlier identification of critically ill children at risk for incurring neurologic morbidity would facilitate heightened surveillance that could lead to timelier clinical detection, earlier interventions, and preserved neurodevelopmental trajectory.
Objective:
Develop machine-learning models for identifying acquired neurologic morbidity while hospitalized with critical illness and assess correlation with contemporary serum-based, brain injury-derived biomarkers.
Design:
Retrospective cohort study.
Setting:
Two large, quaternary children's hospitals.
Exposures:
Critical illness.
Main Outcomes And Measures:
The outcome was neurologic morbidity, defined according to a computable, composite definition at the development site or an order for neurocritical care consultation at the validation site. Models were developed using varying time windows for temporal feature engineering and varying censored time horizons prior to identified neurologic morbidity. Optimal models were selected based on F1 scores, cohort sizes, calibration, and data availability for eventual deployment. A generalizable created at the development site was assessed at an external validation site and optimized with spline recalibration. Correlation was assessed between development site model predictions and measurements of brain biomarkers from a convenience cohort.
Results:
After exclusions there were 14,222-25,171 encounters from 2010-2022 in the development site cohorts and 6,280-6,373 from 2018-2021 in the validation site cohort. At the development site, an extreme gradient boosted model (XGBoost) with a 12-hour time horizon and 48-hour feature engineering window had an F1-score of 0.54, area under the receiver operating characteristics curve (AUROC) of 0.82, and a number needed to alert (NNA) of 2. A generalizable XGBoost model with a 24-hour time horizon and 48-hour feature engineering window demonstrated an F1-score of 0.37, AUROC of 0.81, AUPRC of 0.51, and NNA of 4 at the validation site. After recalibration at the validation site, the Brier score was 0.04. Serum levels of the brain injury biomarker glial fibrillary acidic protein measurements significantly correlated with model output (rs=0.34; P=0.007).
Conclusions And Relevance:
We demonstrate a well-performing ensemble of models for predicting neurologic morbidity in children with biomolecular corroboration. Prospective assessment and refinement of biomarker-coupled risk models in pediatric critical illness is warranted.
