Prediction cardiovascular deterioration in a paediatric intensive care unit (PicEWS): a machine learning modelling
Dan Fredman Stein1,2, Michael J Carter3, John Booth2,4
1Faculty of Life Sciences and Medicine, King's College London, UK.
Insights
Machine learning models can predict cardiovascular deterioration in critically ill children with 90% accuracy, outperforming current clinical scores. This new early warning score (PicEWS) utilizes patient data variability and age-normalisation for improved early detection.
Area of Science:
- Critical Care Medicine
- Biomedical Engineering
- Data Science
Background:
- Paediatric intensive care relies on complex data for high-stakes decisions, often challenged by interpreting longitudinal and subtle vital sign changes.
- Machine learning (ML) offers potential to enhance patient deterioration identification, but prior work often lacked time-series data or age normalization.
- Existing ML models primarily focus on mortality prediction, missing opportunities to identify treatable clinical inflection points like cardiovascular decline.
Purpose of the Study:
- To develop and validate an ML-based early warning score (PicEWS) for predicting cardiovascular deterioration in paediatric intensive care units (PICUs).
- To compare the performance of ML models against the established paediatric Sequential Organ Failure Assessment (pSOFA) score.
- To identify key clinical variables and data features (e.g., trend, variability) crucial for early detection of critical illness.
Main Methods:
- Extracted Electronic Health Record (EHR) data (telemetry, labs, demographics) from 1167 PICU patients.
- Engineered features using a generalizable pipeline, incorporating trend, variability, and age-normalization.
- Compared XGBoost, neural networks, and logistic regression for predicting cardiovascular deterioration within 12 hours, defining the outcome using pSOFA, lactate, ECMO, or death.
Main Results:
- XGBoost achieved the highest performance, with the developed PicEWS predicting cardiovascular deterioration 90% of the time with a low false alarm rate (AUPRC=0.552, AUROC=0.949).
- PicEWS significantly outperformed pSOFA (AUPRC=0.150, AUROC=0.715), which had over 10 false alarms per true alarm.
- Key predictors included blood pressure, bilirubin, COMFORT score, and crucially, feature variability over time.
Conclusions:
- The developed PicEWS demonstrates superior performance in predicting cardiovascular deterioration compared to current clinical standards.
- The study highlights the clinical applicability of ML models using age-stratified data and feature variability for early detection of critical illness in children.
- This approach provides a valuable decision-support tool for improving clinical practice and patient outcomes in paediatric intensive care.
Background:
Paediatric intensive care medicine uses fine granular clinical data that describe substantial patient instability to make high-consequence decisions. However, these decisions are also hindered by clinical experts' ability to interpret longitudinal data along with recent and gradual changes in the vital sign data. Machine learning aided decisions can improve the identification of patient deterioration. Important prior work has predicted outcomes in paediatric intensive care units (PICUs), but has often used non-time series data without age normalisation. Most current work also aims to predict mortality, not potentially treatable clinical inflection points such as cardiovascular deterioration.
Methods:
We extracted telemetry data, alongside laboratory and demographic data, from the Electronic Health Record (EHR) of patients admitted to the general PICU at Great Ormond Street Hospital, London (UK), between 1st April 2019 and 31st April 2021. We engineered deterioration monitoring variables into a smaller feature set using a generalisable pipeline. We calculated trend and variability, and used validated age-normalisation for input variables where appropriate. We compared neural network models, gradient-boosted decision trees (XGBoost), and a logistic regression model to predict cardiovascular deterioration within 12 h (defined as a rise in the paediatric Sequential Organ Failure Assessment (pSOFA) cardiovascular sub-score, rising plasma lactate if lactate ≥2 mmol/l, new extra-corporeal membrane oxygenation (ECMO) requirement, or death). We trained the models on a 70-15-15 percent train-test-validation split. We compared model compositions, including without trend, variability, and frequency of input to smaller models. We investigated feature importance using internal feature importance and Shapley Additive Explanation values. We compared the resulting paediatric intensive care early warning score (PicEWS) with the paediatric Sequential Organ Failure Assessment (pSOFA) score as the gold-standard.
Findings:
1167 patients were included out of a possible 1195. The best performing predictive model for PicEWS was XGBoost. PicEWS was able to predict cardiovascular deterioration 90% of the time, with fewer than two false alarms for every true alarm. For this model, the area under the precision-recall curve (AUPRC) was 0.552, and area under the receiver operator curve (AUROC) was 0.949. This outperformed pSOFA, which yielded over 10 false alarms per true alarm, with an AUPRC of 0.150 and AUROC of 0.715. The most important features for PicEWS included blood pressure, physiological markers such as bilirubin, and COMFORT score (a sedation and behavioural score used in paediatric intensive care). Feature variability was key to model performance. We demonstrated predictions on an individual patient to show model utility. The study showed that machine learning models can outperform current clinical best practice approaches. We use our model to provide insights into future improvements in clinical practice.
Interpretation:
PicEWS outperforms current clinical modelling approaches to predict cardiovascular deterioration. The proposed data processing pipeline and machine learning method offer a clinically applicable decision-support model using age-stratified normal ranges and feature variability over time for the early detection of clinical deterioration in critically ill children.
Funding:
The NIHR Great Ormond Street Biomedical Research Centre at UCL and the Great Ormond Street Hospital Children's Charity peer-reviewed grant award.
