Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model

Area of Science:

• Clinical Informatics
• Critical Care Medicine
• Health Services Research

Background:

• The Epic Sepsis Model version 2 (ESM v2) is a widely used sepsis prediction tool.
• External validation is crucial to guide its adoption and use in diverse clinical settings.

Purpose of the Study:

• To perform a multicenter validation of the ESM v2.
• To compare ESM v2 performance against the earlier ESM v1.
• To assess ESM v2 performance relative to clinician recognition of sepsis.

Main Methods:

• A prognostic study involving adult inpatient encounters across 4 large US health systems.
• Data collected post-implementation of the new model, analyzed using Sepsis-3 criteria.
• Model discrimination assessed via AUROC; performance compared to clinician recognition using order data.

Main Results:

• Encounter-level AUROC for ESM v2 ranged from 0.82 to 0.92 across sites.
• Prediction-level AUROC (12-hour horizon) ranged from 0.75 to 0.85.
• ESM v2 showed slightly decreased performance compared to clinician recognition, with low positive predictive values (0.13-0.26).

Conclusions:

• Multicenter validation of ESM v2 demonstrated improved early sepsis prediction discrimination.
• Significant institutional variability, low positive predictive value, and high alert burden were observed.
• Findings highlight the need for careful consideration of model performance and alert management in clinical practice.