Identifying Adverse Drug Events in Clinical Text Using Fine-Tuned Clinical Language Models: Machine Learning Study

Area of Science:

• Natural Language Processing (NLP) in Healthcare
• Clinical Informatics
• Pharmacovigilance

Background:

• Adverse drug events (ADEs) are a significant healthcare concern, often undocumented in structured electronic health records (EHRs).
• Manual review of clinical notes for ADE detection is labor-intensive and time-consuming.
• Automated extraction of ADE information from clinical text is needed for efficient detection.

Purpose of the Study:

• To fine-tune the Swedish Deidentified Clinical Bidirectional Encoder Representations from Transformers (SweDeClin-BERT) model for Named Entity Recognition (NER) and Relation Extraction (RE).
• To implement an integrated NER-RE pipeline for improved ADE identification in Swedish clinical notes.
• To compare the performance of the fine-tuned model against traditional machine learning methods (CRFs and RF).

Main Methods:

• A subset of clinical notes with suspected ADEs (ICD-10 codes A.1, A.2) from the Stockholm EPR Corpus (2009-2010) was sampled and annotated.
• The SweDeClin-BERT model was fine-tuned for NER and RE tasks.
• An integrated NER-RE pipeline was developed and evaluated on 395 clinical notes, followed by an error analysis.

Main Results:

• 62% of notes contained explicit ADE descriptions, highlighting limitations of relying solely on ICD-10 codes.
• The fine-tuned SweDeClin-BERT achieved high performance: F1-score of 0.845 for NER and 0.81 for RE.
• The integrated NER-RE pipeline achieved an overall F1-score of 0.81, significantly outperforming baseline models.

Conclusions:

• Domain-specific language models like SweDeClin-BERT significantly improve ADE detection in clinical notes compared to conventional machine learning.
• The fine-tuned model shows promise but requires further validation across different hospital settings for generalizability.
• Recommendations include revising annotation guidelines and addressing challenges with compound/split entities in Swedish.