Validation of a Sequential Organ Failure Assessment Score using Electronic Health Record Data

Area of Science:

• Critical Care Medicine
• Health Informatics
• Medical Data Analysis

Background:

• The Sequential Organ Failure Assessment (SOFA) score is vital for assessing illness severity in critical care.
• Electronic health records (EHRs) offer potential for automated SOFA score calculation, expanding its clinical utility.
• Manual validation is crucial to ensure the accuracy of automated clinical scoring systems.

Purpose of the Study:

• To manually validate an existing automated SOFA score system developed using EHR data.
• To assess the agreement between manually and automatically calculated SOFA scores and their components.
• To compare the predictive performance of manual and automated SOFA scores for 30-day in-hospital mortality.

Main Methods:

• Retrospective analysis of 300 critically ill adult patients from a prior randomized trial.
• Manual validation of SOFA scores using EHR data at intensive care unit admission.
• Statistical analysis including Spearman's rank correlation, weighted Cohen's kappa, and Bland-Altman plots.
• Receiver operating characteristic (ROC) curve analysis to evaluate mortality prediction.

Main Results:

• High agreement was observed between manual and automated SOFA scores (Spearman's rho = 0.90).
• The renal and respiratory components showed lower agreement compared to the overall score.
• Both manual (AUC = 0.77) and automated (AUC = 0.75) SOFA scores demonstrated similar predictive value for 30-day in-hospital mortality.

Conclusions:

• Automated SOFA score calculation from EHR data is feasible and reliable.
• The automated system highly correlates with manual assessments, supporting its clinical applicability.
• Automated SOFA scores maintain comparable prognostic accuracy for mortality in critically ill patients.