Enhancing quality of antimicrobial prescribing through 'Ask Eolas' (language model): a user-testing and simulation...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems

Background:

• Existing antimicrobial guidance tools have limitations in accuracy and usability.
• Antimicrobial stewardship programs require effective tools to reduce prescribing errors.

Purpose of the Study:

• To compare the prescribing accuracy, error reduction, usability, and clinician confidence of Ask Eolas against existing antimicrobial guidance tools.
• To evaluate the performance of a retrieval-augmented generation-enhanced AI-CDSS in a simulated prescribing environment.

Main Methods:

• A structured, single-site simulation study was conducted with 45 healthcare professionals.
• Participants evaluated Ask Eolas and two comparator groups (Eolas App, PDF Guidelines) across 45 prescribing cases.
• Prescribing accuracy, error rates, usability, and clinician confidence were assessed.

Main Results:

• Ask Eolas achieved zero prescribing errors, compared to six and eight errors in the comparator groups (p < 0.001).
• The number needed to treat with Ask Eolas was 1.9, indicating one additional error-free prescription for every two clinicians using it.
• Ask Eolas demonstrated significant improvements in prescribing accuracy, usability, clinician confidence, and system transparency.

Conclusions:

• Ask Eolas, an AI-CDSS, significantly enhances prescribing accuracy and clinician confidence compared to traditional tools.
• The findings support the safe deployment of AI-CDSS according to the TRUST-AI framework.
• Further real-world implementation studies are recommended for antimicrobial stewardship programs.