Related Experiment Video
Updated: May 29, 2025
Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale
Published on: August 25, 2014
Interoperable Models for Identifying Critically Ill Children at Risk of Neurologic Morbidity
Christopher M Horvat1,2, Amie J Barda3, Eddie Perez Claudio4
1Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania.
Insights
Machine learning models can identify neurologic morbidity in critically ill children, aiding early detection and intervention. Biomarker correlation supports model accuracy for improved neurodevelopmental outcomes.
Area of Science:
- Pediatric Critical Care Medicine
- Neuroscience
- Artificial Intelligence in Healthcare
Background:
- Decreased pediatric critical care mortality shifts focus to neurodevelopmental outcomes.
- Early identification of neurologic morbidity is crucial for timely interventions.
- Developing predictive models can enhance surveillance for at-risk children.
Purpose of the Study:
- To develop machine-learning models to identify acquired neurologic morbidity in critically ill children.
- To assess the correlation between these models and serum-based brain injury biomarkers.
- To improve neurodevelopmental potential preservation in pediatric intensive care.
Main Methods:
- A prognostic study utilizing data from two quaternary pediatric intensive care units (development and external validation).
- Machine learning models (extreme gradient boosted) were developed and validated.
- Neurologic morbidity was the primary outcome, defined by composite criteria or neurocritical care consultation.
- Correlation with serum biomarkers (glial fibrillary acidic protein) was assessed.
Main Results:
- A generalizable model achieved an F1 score of 0.37 and AUC of 0.81 at the validation site.
- The number needed to alert was 4, with a Brier score of 0.04 after recalibration.
- Serum glial fibrillary acidic protein levels showed significant correlation with model predictions (rs=0.34, P=.007).
Conclusions:
- The study demonstrates a well-performing machine learning model for predicting neurologic morbidity in critically ill children.
- Biomolecular corroboration supports the model's clinical relevance.
- Further prospective validation and refinement of these biomarker-coupled risk models are warranted.
Importance:
Decreasing mortality in the field of pediatric critical care medicine has shifted practicing clinicians' attention to preserving patients' neurodevelopmental potential as a main objective. Earlier identification of critically ill children at risk for incurring neurologic morbidity would facilitate heightened surveillance that could lead to timelier clinical detection, earlier interventions, and preserved neurodevelopmental trajectory.
Objectives:
To develop machine-learning models for identifying acquired neurologic morbidity in hospitalized pediatric patients with critical illness and assess correlation with contemporary serum-based, brain injury-derived biomarkers.
Design, Setting, And Participants:
This prognostic study used data from all children admitted to a quaternary pediatric intensive care unit in a large, freestanding children's hospital in Western Pennsylvania between January 1, 2010, and December 31, 2022. External model validation used data from children admitted between January 1, 2018, and December 31, 2023, to a quaternary pediatric intensive care unit in a large, freestanding children's hospital that serves as a referral center for the 5-state region of Washington, Wyoming, Alaska, Montana, and Idaho.
Exposures:
Critical illness.
Main Outcomes And Measures:
The outcome was neurologic morbidity, defined according to a computable, composite definition at the development site or an order for neurocritical care consultation at the validation site. Models were developed using varying time windows for temporal feature engineering and varying censored time horizons between the last feature and the identified neurologic morbidity. A generalizable model created at the development site was optimized and assessed at an external validation site. Correlation was assessed between development site model predictions and measurements of brain biomarkers from a convenience cohort.
Results:
After exclusions, there were 18 568 encounters from 2010 to 2022 in the development site generalizable model cohort (median age, 70 [IQR, 18-161] months; 8325 [45%] female). There were 6825 encounters from 2018 to 2021 at the external validation site (median age, 96 [IQR 18-171] months; 3159 [46%] female). A generalizable extreme gradient boosted model with a 24-hour time horizon and 48-hour feature engineering window demonstrated an F1 score of 0.37 (95% CI, 0.33-0.40), area under the receiver operating characteristics curve of 0.81 (95% CI, 0.78-0.83), and number needed to alert of 4 at the validation site. After recalibration at the validation site, the Brier score was 0.04. Serum levels of the brain injury biomarker glial fibrillary acidic protein significantly correlated with model output (rs = 0.34; P = .007).
Conclusions And Relevance:
This prognostic study of prediction models for detecting neurologic morbidity in critically ill children demonstrated a well-performing ensemble of models with biomolecular corroboration. Prospective assessment and refinement of biomarker-coupled risk models in pediatric critical illness are warranted.
