Enhancing antimicrobial stewardship using a retrieval-augmented generation large language model for infectious...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Pharmacology

Background:

• Antimicrobial resistance necessitates adherence to treatment guidelines.
• Standardizing antimicrobial therapy is crucial for effective patient care.
• Existing methods for protocol adherence can be inconsistent.

Purpose of the Study:

• To develop and evaluate a retrieval-augmented generation (RAG)-based large language model system.
• To standardize and improve adherence to antimicrobial treatment protocols.
• To leverage the Antimicrobial Treatment Guide from Hospital Israelita Albert Einstein.

Main Methods:

• A four-module system: document processing, query processing, large language model, and prompt management.
• Manual segmentation of protocol documents into text chunks, vector conversion, and storage in a vector database.
• Evaluation using Jaccard index for retrieval accuracy and SUS usability scale.

Main Results:

• The RAG system achieved an excellent usability score (SUS) of 82.
• The system correctly retrieved the appropriate antimicrobial protocol in 83% of cases.
• Demonstrated effective alignment between system responses and institutional clinical guidelines.

Conclusions:

• The RAG-based system offers high usability and reliable retrieval for standardized antimicrobial therapy.
• This technology can bridge the gap between clinical practice and protocol adherence.
• Potential to enhance diagnostic accuracy, treatment specificity, reduce antibiotic resistance, and improve patient outcomes.