Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

Area of Science:

• Emergency Medicine
• Data Science in Healthcare
• Clinical Decision Support

Background:

• Accurate sepsis identification is vital for emergency department (ED) patient triage and care.
• Existing sepsis identification models often focus on ICU patients, with limited external validation.
• Discrepancies in model performance across different datasets are a significant challenge.

Purpose of the Study:

• To develop and externally validate a machine learning (ML) model for sepsis patient stratification in the ED.
• To compare the performance of the ML model against traditional clinical tools like quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS).

Main Methods:

• Retrospective collection of clinical data from two geographically distinct institutes.
• Development of an eXtreme Gradient Boosting (XGBoost) algorithm for sepsis identification.
• External validation using Sepsis-3 criteria as the reference standard.

Main Results:

• The XGBoost model achieved an AUROC of 0.86 in internal validation, significantly outperforming SIRS (0.68) and qSOFA (0.56).
• External validation showed a reduced AUROC of 0.75 for XGBoost, though still superior to SIRS (0.57) and qSOFA (0.66).
• Model performance varied due to heterogeneity in patient characteristics like prevalence, severity, age, and comorbidities.

Conclusions:

• The developed ML model demonstrates strong discriminative capabilities for sepsis identification in the ED.
• While outperforming existing tools, dataset discrepancies necessitate careful evaluation before clinical implementation.
• External validation is essential to ensure the generalizability and reliability of ML models in diverse healthcare settings.