Improving sepsis best practice utility and clinical acceptance using an LLM-enhanced prediction system

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems

Background:

• Early sepsis detection is critical for reducing mortality but faces diagnostic challenges due to non-specific symptoms.
• Current sepsis prediction systems often lack contextual understanding of patient data, leading to diagnostic delays.

Purpose of the Study:

• To evaluate the impact of a large language model (LLM)-enhanced sepsis prediction system (COMPOSER-LLM) on clinical perception and acceptance of best practice advisories (BPAs).
• To assess the perceived utility and impact of the COMPOSER-LLM BPA on nursing care and sepsis risk assessment.

Main Methods:

• Implementation of COMPOSER-LLM, integrating an LLM to analyze clinical notes for sepsis risk assessment in emergency departments.
• Bayesian causal impact analysis to evaluate BPA relevance and clinical acceptance (using "No Infection Suspected" acknowledgements as a proxy).
• Post-deployment surveys to assess nurses' perceptions of the BPA's utility and impact on patient care.

Main Results:

• A significant decrease in "No Infection Suspected" acknowledgements post-deployment, indicating higher BPA clinical acceptance.
• Survey results showed nurses found the COMPOSER-LLM BPA useful, aiding in identifying at-risk patients and building trust.
• Nurses with 0-5 years of experience were more likely to perceive infection risk and increased sepsis expectation after BPA activation.

Conclusions:

• LLM augmentation in sepsis prediction systems can enhance BPA clinical acceptance and nurse trust.
• COMPOSER-LLM demonstrates potential for improving early sepsis detection by providing contextual support for risk assessment.
• Further investigation into experience-based differences in BPA interpretation may refine implementation strategies for sepsis prediction tools.