Subtle variation in sepsis-III definitions markedly influences predictive performance within and across methods.

Area of Science:

• Medical Informatics
• Clinical Decision Support
• Artificial Intelligence in Healthcare

Background:

• Early sepsis detection is critical for effective clinical intervention.
• Limited availability of end-to-end pipelines hinders direct comparison of sepsis prediction methodologies.
• Inconsistent reconstruction of sepsis onset time impedes research progress.

Purpose of the Study:

• To evaluate the impact of different sepsis onset definitions on predictive model performance.
• To compare the variability in model performance due to onset definition versus model architecture.
• To highlight the need for standardized sepsis onset definitions in research.

Main Methods:

• Retrospective cohort study using the MIMIC-III database.
• Analysis of predictive model performance (tree-based, deep learning, survival analysis) under three sepsis onset definitions derived from sepsis-III criteria.
• Comparison of model performance variations attributed to onset definition versus inherent model differences.

Main Results:

• Model performance showed greater sensitivity to variations in sepsis onset definition than to the specific predictive model used.
• The choice of onset time definition resulted in a 0-6% variation in area under the receiver operating characteristic (AUROC).
• Differences in model performance were marginal (1-5% AUROC gain) when a fixed sepsis definition was applied.

Conclusions:

• The definition of sepsis onset significantly influences the performance metrics of predictive models.
• Comparing predictive models without accounting for the sepsis definition can lead to erroneous conclusions.
• Standardization of sepsis onset definitions is essential for reproducible and reliable sepsis prediction research.