A Large Language Model for Extracting Post-marketing Adverse Drug Events from Clinical Notes in the Electronic Health...

Area of Science:

• Pharmacovigilance and Drug Safety
• Natural Language Processing in Healthcare
• Clinical Informatics

Background:

• Clinical notes contain valuable adverse drug event (ADE) data often missed by traditional methods.
• Existing rule-based or sentence-level models struggle with subtle causal cues and generate many false positives.

Purpose of the Study:

• To develop a large language model (LLM) pipeline for processing electronic health record (EHR) notes.
• To identify drug-event pairs with a plausible causal link and infer ADE properties like seriousness and label status.

Main Methods:

• A two-pass LLM workflow was employed, with Pass 1 screening for ADEs and Pass 2 adding structured fields.
• Expert review and a gold standard dataset were used to validate ADEs, assess seriousness, and determine label status.
• The study utilized 372 deidentified EHR notes from diverse clinical settings.

Main Results:

• The LLM achieved 94.2% precision and 84.1% recall in identifying ADEs.
• Seriousness and label status were correctly inferred in 100% and 93.9% of cases, respectively.
• The cost of inference was low ($0.18 per note), and the model identified ADEs not explicitly mentioned by clinicians.

Conclusions:

• LLM-processed EHR notes can transform free-text data into actionable ADE information with high accuracy.
• This approach can uncover new safety signals for potential drug side effects.
• Integration with pharmacovigilance platforms can accelerate the detection of rare, serious, and unlabeled ADEs for regulatory review.