Leveraging Large Language Models in Extracting Drug Safety Information from Prescription Drug Labels

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Pharmacovigilance

Background:

• Adverse drug reactions (ADRs) and drug interactions are significant causes of morbidity and mortality.
• Structured Product Labels (SPLs) are a key source of drug safety data.
• Extracting safety information from SPLs using traditional NLP methods is challenging.

Purpose of the Study:

• To evaluate generative language models (LLMs) for extracting drug safety information from SPLs.
• To compare the performance of different LLMs against baseline methods.
• To assess the adaptability of LLMs for retrieving drug interaction data.

Main Methods:

• Compared GPT, Llama, and Mixtral LLMs against two baseline methods for adverse reaction (AR) extraction from SPLs.
• Investigated the impact of prompting strategies and term complexity on AR extraction.
• Assessed the ability of generative models to extract drug interactions without fine-tuning.

Main Results:

• Generative LLMs, particularly GPT-4, achieved performance comparable to or better than state-of-the-art models without additional training.
• Extraction performance was influenced by SPL section, context, and AR term complexity.
• Demonstrated model generalizability by extracting drug names from the drug interaction section.

Conclusions:

• Generative LLMs show strong potential for automating drug safety information extraction from SPLs.
• This automation can enhance post-market surveillance and contribute to reducing ADRs.
• Future research should refine prompting and expand model capabilities for complex safety data.