Pharmacovigilance with Transformers: A Framework to Detect Adverse Drug Reactions Using BERT Fine-Tuned with FARM

Area of Science:

• Pharmacovigilance
• Natural Language Processing
• Computational Linguistics

Background:

• Adverse drug reactions (ADRs) are significant health concerns.
• Social media platforms are increasingly used for sharing health-related information, including ADR experiences.
• Existing methods for ADR detection from text face challenges in efficiency and end-to-end system development.

Purpose of the Study:

• To develop an end-to-end system for modeling and detecting adverse drug reactions (ADRs) from natural language text.
• To improve the computational efficiency of ADR detection systems compared to standard BERT models.
• To integrate text classification and ADR mention extraction into a unified, faster framework.

Main Methods:

• Fine-tuning BERT using the Framework for Adapting Representation Models (FARM) to create a FARM-BERT model.
• Implementing a multitask learning approach with multiple prediction heads for text classification and ADR sequence labeling.
• Evaluating the system on diverse datasets including Twitter, PubMed, TwiMed-Twitter, and TwiMed-PubMed.

Main Results:

• The FARM-BERT model achieved high F-scores across all tested datasets: 89.6% (Twitter), 97.6% (PubMed), 84.9% (TwiMed-Twitter), and 95.9% (TwiMed-PubMed).
• The proposed model demonstrated significantly faster training and testing times compared to the standard BERT model.
• The modular FARM framework facilitated easier and computationally faster training of the end-to-end ADR extraction system.

Conclusions:

• FARM-BERT offers a computationally efficient and effective solution for end-to-end ADR detection from text.
• The multitask learning approach within the FARM framework streamlines the development and improves the performance of ADR extraction systems.
• This approach holds promise for real-world applications in pharmacovigilance and public health monitoring.