Related Experiment Video
Updated: Jun 27, 2025
Observational Study Protocol for Repeated Clinical Examination and Critical Care Ultrasonography Within the Simple Intensive Care Studies
Published on: January 16, 2019
Automated Calculator for the Pediatric Sequential Organ Failure Assessment Score: Development and External Validation
Alireza Akhondi-Asl1,2,3,4, Matthew Luchette1,2,3,4, Nilesh M Mehta1,2,3,4
1Division of Critical Care Medicine, Department of Anesthesiology, Critical Care & Pain Medicine, Boston Children's Hospital, Boston, MA.
Objectives:
The pediatric Sequential Organ Failure Assessment (pSOFA) score summarizes severity of organ dysfunction and can be used to predict in-hospital mortality. Manual calculation of the pSOFA score is time-consuming and prone to human error. An automated method that is open-source, flexible, and scalable for calculating the pSOFA score directly from electronic health record data is desirable.
Design:
Single-center, retrospective cohort study.
Setting:
Quaternary 40-bed PICU.
Patients:
All patients admitted to the PICU between 2015 and 2021 with ICU stay of at least 24 hours.
Interventions:
None.
Measurements And Main Results:
We used 77 records to evaluate the automated score. The automated algorithm had an overall accuracy of 97%. The algorithm calculated the respiratory component of two cases incorrectly. An expert human annotator had an initial accuracy of 75% at the patient level and 95% at the component level. An untrained human annotator with general clinical research experience had an overall accuracy of 16% and component-wise accuracy of 67%. Weighted kappa for agreement between the automated method and the expert annotator's initial score was 0.92 (95% CI, 0.88-0.95), and between the untrained human annotator and the automated score was 0.50 (95% CI, 0.36-0.61). Data from 9146 patients (in-hospital mortality 3.6%) were included to validate externally the discriminability of the automated pSOFA score. The admission-day pSOFA score had an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.82).
Conclusions:
The developed automated algorithm calculates pSOFA score with high accuracy and is more accurate than a trained expert rater and nontrained data abstracter. pSOFA's performance for predicting in-hospital mortality was lower in our cohort than it was for the originally derived score.
