Aligning prediction models with clinical information needs: infant sepsis case study.

Area of Science:

• Neonatal Medicine
• Machine Learning in Healthcare
• Clinical Decision Support

Background:

• Sepsis recognition in Neonatal Intensive Care Units (NICUs) is complex, and delayed diagnosis has severe outcomes.
• Existing predictive models for sepsis have faced limited clinical adoption.
• Aligning machine learning (ML) models with clinician information needs is crucial for improving sepsis outcomes.

Purpose of the Study:

• To develop and compare a two-component ML pipeline (baseline and dynamic risk) against a single ML model for neonatal sepsis prediction.
• To evaluate if a pipeline model mirroring clinician reasoning improves sepsis detection.
• To assess model performance 1 hour prior to clinical sepsis recognition.

Main Methods:

• Developed logistic regression and XGBoost models using electronic healthcare record data from an NICU cohort.
• Utilized a two-stage pipeline model and a single integrated model.
• Employed nested 10-fold cross-validation for performance evaluation.

Main Results:

• The single XGBoost model demonstrated superior performance with a sensitivity of 0.77 and specificity of 0.83.
• The two-stage pipeline XGBoost model achieved a sensitivity of 0.72 and specificity of 0.84.
• Both models predicted sepsis risk 1 hour prior to clinical recognition.

Conclusions:

• A single ML model outperformed a two-stage pipeline designed to align with clinical reasoning for neonatal sepsis prediction.
• The pipeline model showed inferior sensitivity compared to the single model at similar specificity levels.
• Further research is necessary to enhance the alignment of ML model outputs with clinician information needs in the NICU.