A Multi-Phase Analysis of Blood Culture Stewardship: Machine Learning Prediction, Expert Recommendation Assessment,...

Area of Science:

• * Clinical Informatics
• * Machine Learning in Healthcare
• * Diagnostic Stewardship

Background:

• * Overordering of blood cultures strains resources and fuels antimicrobial resistance.
• * Global antibiotic shortages exacerbate the need for optimized diagnostic practices.

Purpose of the Study:

• * To develop and evaluate machine learning (ML) models for predicting bacteremia risk in emergency department (ED) patients.
• * To compare the performance of ML models against expert recommendation frameworks and large language model (LLM) pipelines.

Main Methods:

• * Development of ML models using structured electronic health record (EHR) data and unstructured provider notes from 135,483 ED blood culture orders.
• * Integration of natural language processing (NLP) via LLM for analyzing provider notes.
• * Evaluation of model performance using Area Under the Curve (AUC), sensitivity, and specificity.

Main Results:

• * ML models integrating structured EHR data and provider notes achieved an AUC of 0.81.
• * The ML approach demonstrated superior specificity (without compromising sensitivity) compared to expert recommendations and an LLM pipeline.
• * The LLM pipeline achieved high sensitivity (96%) but significantly lower specificity (16%).

Conclusions:

• * ML models combining structured and unstructured data provide a more accurate and specific method for predicting bacteremia risk.
• * This data-driven approach enhances diagnostic stewardship beyond current standards of care.
• * Improved prediction can lead to more judicious blood culture ordering and appropriate antibiotic use.