Extracting and Classifying Drug Discontinuations From Estonian Electronic Health Records: Development and Validation...

Area of Science:

• Pharmacovigilance
• Health Informatics
• Natural Language Processing

Background:

• Medication adherence is critical for managing chronic diseases, but treatment discontinuation is common due to side effects, inefficacy, or cost.
• Reasons for discontinuation are often buried in free-text clinical notes, hindering large-scale analysis.
• Analyzing unstructured data in low-resource languages like Estonian presents unique challenges for traditional methods.

Purpose of the Study:

• To evaluate the capability of large language models (LLMs) in extracting and classifying reasons for drug discontinuation from Estonian electronic health records.
• To determine whether patients or clinicians initiated the discontinuation.
• To characterize discontinuation patterns and initiators for statin and antidiabetic medications.

Main Methods:

• Utilized prescription data and free-text anamneses from a 10% sample of the Estonian population (2012-2019).
• Applied LLMs (Llama 3.1-70B and GPT-4o) to extract discontinuation phrases/reasons and classify them.
• Assessed LLM performance on 100 randomly selected cases per drug group for extraction and classification accuracy.

Main Results:

• Identified 625 antidiabetic drug and 233 statin discontinuation cases.
• Achieved high precision (0.93-0.98) for phrase extraction and (0.95-0.96) for reason extraction.
• Weighted F1-scores for reason classification ranged from 0.81-0.84, and for initiator classification from 0.64-0.78. Adverse reactions were the most frequent reason for statin (70%) and antidiabetic (44.8%) discontinuations.

Conclusions:

• LLMs demonstrate strong accuracy in extracting and classifying medication discontinuation reasons in Estonian clinical narratives.
• LLMs show variable but promising performance in identifying discontinuation initiators, enabling scalable analysis.
• This approach unlocks valuable insights from unstructured clinical notes, aiding in monitoring treatment patterns and detecting adverse event signals.