Performance of a Sepsis Prediction Model Across Different Sepsis Definitions.

Area of Science:

• Clinical Informatics
• Healthcare AI
• Sepsis Prediction

Background:

• Early sepsis detection is crucial for patient outcomes.
• Previous sepsis models, like the Early Detection of Sepsis Model v1, have demonstrated suboptimal performance.
• Comparing sepsis models is challenging due to diverse outcome definitions and limited generalizability.

Purpose of the Study:

• To evaluate the performance of the Early Detection of Sepsis Model, version 2.
• To assess the model's efficacy using multiple standard sepsis definitions, including Sepsis-3, SEP-1, and ASE.

Main Methods:

• A diagnostic study included nearly 200,000 adult patient encounters across 9 hospitals.
• A gradient-boosted tree ensemble model, incorporating demographics, vitals, labs, and medications, generated predictions every 15 minutes.
• Model performance was evaluated against Sepsis-3, SEP-1, and ASE definitions using AUROC and AUPRC metrics.

Main Results:

• The model demonstrated moderate predictive performance across definitions, with AUROC ranging from 0.85 to 0.94.
• Area Under the Precision-Recall Curve (AUPRC) varied significantly, indicating challenges with precision (0.11-0.24).
• Lead times for sepsis detection ranged from 1.4 to 4.5 hours, with positive predictive values (PPV) between 5.9% and 11.4%.

Conclusions:

• The locally trained sepsis model exhibits moderate predictive capabilities, but performance varies substantially based on the sepsis definition used.
• The model can provide advance warning of sepsis onset.
• A high false-positive rate may limit clinical utility, underscoring the need for careful threshold selection and tailored implementation strategies.