Large language models in pharmacist-led direct patient care: A rapid review of published evaluations

Large language models (LLMs) represent an advancement in artificial intelligence technology with the potential to support patient care tasks. However, reviews of their evaluations have often excluded pharmacy-specific applications.

To characterize published evaluations of LLMs for pharmacist-led direct patient care tasks, synthesize reported LLM performance, and identify opportunities and challenges for workflow integration.

Searches were conducted in Embase, MEDLINE, PubMed, CINAHL, Scopus, and Web of Science for studies published between January 2014 and April 2026 using the concepts "pharmacists," "pharmacy," and "LLMs." Original peer-reviewed studies evaluating LLMs for tasks aligned with pharmacist-led direct patient care were eligible. Covidence was used for screening studies; data were extracted and narratively synthesized.

Of 4925 identified records, 71 studies were included in the synthesis. Pharmacist-led direct patient care tasks were identified for which LLMs were evaluated; accuracy was the most assessed metric across studies (n = 53), and ChatGPT variants were the most widely evaluated LLMs (n = 63). Evaluation and reporting approaches varied across studies, limiting cross-study comparisons; however, studies generally reported higher performance for advanced LLM versions than for earlier ones. Challenges to workflow integration include confabulation, costs, and privacy risks.

Evaluations of LLMs largely focused on ChatGPT models and accuracy-based outcomes; however, heterogeneity in evaluation and reporting limited cross-study comparisons. Advanced LLMs generally outperformed earlier versions, suggesting increasing potential to support pharmacist-led direct patient care tasks. Future evaluations should adopt standardized approaches and include human factors outcomes relevant to workflow integration.