'The Score Matters': wide variations in predictive performance of 18 paediatric track and trigger systems
Susan M Chapman1,2,3, Jo Wray2,4, Kate Oulton2,4
1Great Ormond Street Hospital, London, UK.
Insights
This study compared 18 pediatric early warning systems (PEWS) for predicting critical deterioration in children. Three PEWS showed superior performance, while trigger-based systems were less effective.
Area of Science:
- Pediatric critical care medicine
- Clinical risk prediction
- Health informatics
Background:
- Pediatric Early Warning Systems (PEWS) are crucial for identifying at-risk children.
- Significant variation exists in the predictive performance of published PEWS.
- The choice of PEWS can have important clinical implications.
Purpose of the Study:
- To compare the predictive performance of 18 different PEWS.
- To identify the most effective PEWS for predicting critical deterioration in pediatric patients.
- To evaluate the performance differences between scoring and trigger-based PEWS.
Main Methods:
- Retrospective case-controlled study conducted at a UK tertiary referral children's hospital.
- PEWS values were calculated from existing clinical data for patients admitted between 2011 and 2012.
- Area Under the Receiver Operator Characteristic Curve (AUROC) was used to compare predictive performance.
Main Results:
- Three out of 18 PEWS demonstrated superior predictive performance with an AUROC of 0.87 or greater.
- Significant performance differences were found between PEWS, with 13 systems showing statistically significant variations.
- Trigger-based systems generally performed worse than scoring systems.
Conclusions:
- There is considerable variation in PEWS performance, highlighting the clinical importance of system selection.
- Trigger-based systems were found to perform poorly overall.
- Increased system complexity did not necessarily correlate with improved predictive performance.
Objective:
To compare the predictive performance of 18 paediatric early warning systems (PEWS) in predicting critical deterioration.
Design:
Retrospective case-controlled study. PEWS values were calculated from existing clinical data, and the area under the receiver operator characteristic curve (AUROC) compared.
Setting:
UK tertiary referral children's hospital.
Patients:
Patients without a 'do not attempt resuscitation' order admitted between 1 January 2011 and 31 December 2012. All patients on paediatric wards who suffered a critical deterioration event were designated 'cases' and matched with a control closest in age who was present on the same ward at the same time.
Main Outcome Measures:
Respiratory and/or cardiac arrest, unplanned transfer to paediatric intensive care and/or unexpected death.
Results:
12 'scoring' and 6 'trigger' systems were suitable for comparative analysis. 297 case events in 224 patients were available for analysis. 244 control patients were identified for the 311 events. Three PEWS demonstrated better overall predictive performance with an AUROC of 0.87 or greater. Comparing each system with the highest performing PEWS with Bonferroni's correction for multiple comparisons resulted in statistically significant differences for 13 systems. Trigger systems performed worse than scoring systems, occupying the six lowest places in the AUROC rankings.
Conclusions:
There is considerable variation in the performance of published PEWS, and as such the choice of PEWS has the potential to be clinically important. Trigger-based systems performed poorly overall, but it remains unclear what factors determine optimum performance. More complex systems did not necessarily demonstrate improved performance.
More Related Videos
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
08:22Measuring Cardiac Autonomic Nervous System ANS Activity in Toddlers - Resting and Developmental Challenges
Published on: February 25, 2016