Using Natural Language Processing to Identify Adverse Drug Events Characterized by Medication Replacement in Primary...

Area of Science:

• Health Informatics
• Computational Linguistics
• Pharmacovigilance

Background:

• Healthcare systems generate extensive unstructured clinical notes containing valuable patient information.
• Traditional research relies on structured data, overlooking the nuances in clinical text.
• Natural Language Processing (NLP) offers a method to extract insights from unstructured clinical notes, potentially improving adverse drug event (ADE) detection.

Purpose of the Study:

• To train and evaluate multiple NLP models for identifying ADEs in clinical notes.
• To compare the performance of established NLP architectures against a novel model.
• To assess the impact of data reformatting (SOAP structure) on model performance.

Main Methods:

• Utilized electronic medical records from the Manitoba Primary Care Research Network (MaPCReN).
• Annotated clinical notes for ADEs, associated terms, and drugs in patients aged 55+.
• Trained and evaluated NLP models (BioBERT, BlueBERT, LLM classifiers) on original and SOAP-formatted notes, using precision, recall, and F1-score for evaluation.

Main Results:

• Models trained on SOAP-rewritten notes generally outperformed those on original notes.
• The SOAP Rewrite+ LLM Embeddings Classifier (GritLM-SOAP) achieved the highest mean F1-score of 72.53%.
• BioBERT-SOAP weighted demonstrated the highest mean recall at 76.34%; relation extraction was identified as a bottleneck for end-to-end extraction.

Conclusions:

• NLP models that mimic clinician reasoning can enhance ADE detection in clinical notes.
• While the GritLM-SOAP model shows promise, further advancements are needed for optimal performance.
• The study highlights the potential of NLP in improving pharmacovigilance through analysis of unstructured clinical data.