Large language models for preventing medication direction errors in online pharmacies.

Area of Science:

• Health Informatics
• Artificial Intelligence in Pharmacy
• Patient Safety

Background:

• Medication direction errors, including incorrect dosage or frequency, pose significant patient safety risks, increasing the likelihood of adverse drug events.
• Large language models (LLMs) offer advanced text interpretation and generation capabilities, presenting an opportunity to address these critical errors.

Purpose of the Study:

• To explore the integration of domain knowledge with LLMs to reduce errors in pharmacy medication directions.
• To introduce and evaluate MEDIC (medication direction copilot), a system designed to emulate pharmacist reasoning for precise prescription communication.

Main Methods:

• Developed MEDIC by fine-tuning a first-generation LLM with 1,000 expert-annotated medication directions from Amazon Pharmacy.
• The system extracts core prescription components (dosage, frequency) and reassembles them using pharmacy logic and safety guardrails.
• Compared MEDIC against two LLM-based benchmarks using 1,200 expert-reviewed prescriptions and tested its real-world performance in an online pharmacy.

Main Results:

• MEDIC recorded significantly fewer near-miss events (errors caught before patient administration) compared to two LLM benchmarks: 1.51 and 4.38 times fewer, respectively.
• In a real-world deployment, MEDIC reduced near-miss events by 33% (CI 26%, 40%).
• The study highlights improved accuracy and efficiency in pharmacy operations through domain-expert-enhanced LLMs.

Conclusions:

• LLMs, when augmented with domain expertise and safety measures, can substantially enhance the accuracy and efficiency of pharmacy operations.
• MEDIC demonstrates the potential of specialized AI systems to improve patient safety by minimizing medication direction errors.