Trigger-first large language model-assisted adverse drug reaction surveillance: A pharmacist-supervised retrospective cohort study

Active pharmacovigilance in low-prevalence settings requires review of large electronic health record cohorts. We evaluated a pharmacist-supervised workflow combining electronic triggers with large language model (LLM)-assisted review.

To assess candidate enrichment and stage-specific performance for adverse drug reaction (ADR) surveillance.

In this retrospective cohort study, two pharmacists established a blinded reference standard for 7950 exposure episodes from 7950 unique inpatients. A prespecified Chinese Hospital Pharmacovigilance System trigger selected candidates for Qwen3-4B-Instruct and an institution-specific low-rank adaptation (LoRA)-fine-tuned model trained on 410 patient-disjoint cases involving other medicines. Paired errors were compared using exact McNemar tests.

Twenty-eight episodes were ADR-positive. The trigger selected 167 candidates (2.10%), with 100.0% sensitivity (95% confidence interval, 87.66-100.0%) and 98.25% specificity. For the base versus LoRA-fine-tuned model, sensitivity was 82.14% versus 92.86%, specificity 88.49% versus 94.96%, positive predictive value 58.97% versus 78.79%, and F1 score 68.66% versus 85.25%. LoRA reduced false positives from 16 to 7 (9 corrected, 0 new; P = 0.0039). The numerical increase in sensitivity was not statistically significant (P = 0.250), with only 28 ADR-positive episodes available for evaluation.

Trigger-first screening reduced review volume, while LoRA fine-tuning improved false-positive control. LLM classifications should support pharmacist review and should not be used as an autonomous exclusion rule.