Performance validation of large language models for drug-drug interaction consultations: Potential pharmacist...

The growing prevalence of polypharmacy and time constraints pose significant challenges for pharmacists in managing drug-drug interactions during outpatient consultations. While large language models (LLMs) show potential in various medical applications, their practical utility in real-world pharmaceutical services particularly in providing accurate, context-aware drug-drug interaction advice, remains underexplored.

This study aims to evaluate the performance of LLMs in responding to real-world drug-drug interaction consultation queries. The goal is to assess their feasibility as assistive tools for pharmacists in delivering evidence-based, patient-specific medication guidance.

A total of 165 nonredundant drug-drug interaction queries were extracted from outpatient pharmacy consultations at Beijing Chao-Yang Hospital from January to May 2023. Responses from 4 LLMs were compared against answers provided by pharmacists. A scoring framework was applied to evaluate accuracy, depth of analysis, and practicality of administration advice.

The LLMs demonstrated high consistency with pharmacists' responses, with rates of 84.85%-89.09%. Discrepancies arose primarily from incomplete pharmacist documentation, LLM drug misidentification, and differences in practical medication use advice. In scored evaluations, DeepSeek R1 achieved the highest score, achieving 3-4-point scores in 137 queries by integrating multi-perspective drug analysis and actionable administration strategies.

LLMs, particularly those with advanced reasoning and internet retrieval capabilities, have the potential to support pharmacists in drug-drug interaction management by delivering accurate, context-specific recommendations. However, limitations such as potential hallucination and operational delays remain. LLMs should serve as complementary tools under pharmacist supervision to enhance consultation quality and efficiency, not as replacements for professional judgment. Further validation using clinical metrics is warranted before widespread adoption.