Pre-implementation safety evaluation of an AI decision-support system for surgical antimicrobial prophylaxis

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems
• Antimicrobial Stewardship

Background:

• Large language models (LLMs) offer potential for antimicrobial prescribing support.
• Antimicrobial prescribing is a high-risk area for digital decision-support in Australia.
• Institution-specific safety evaluation is crucial before clinical LLM implementation.

Purpose of the Study:

• To assess LLM accuracy, error types, and antimicrobial stewardship risks.
• The study focused on LLM performance using public surgical prophylaxis guidelines.
• Evaluation was conducted across 20 simulated surgical scenarios.

Main Methods:

• Twenty simulated surgical scenarios were used to test a LLM.
• The LLM received public surgical prophylaxis guidelines during inference.
• Outputs (agent, dose, timing, re-dosing, citation) were assessed by clinicians for accuracy and harm.

Main Results:

• Clinically significant antimicrobial prescribing risks were found in 10% of scenarios.
• Identified risks included missed anaerobic coverage and inadequate prophylaxis re-dosing.
• Guideline concordance was high (4/5), but timing (70%) and citation (45%) accuracy were lower.

Conclusions:

• Guideline-based AI systems using LLMs are feasible to construct.
• Stewardship-relevant safety risks were identified, preventing current clinical use.
• Further validation is necessary before LLMs can be safely implemented in antimicrobial prescribing.