MultiADE: A Multi-domain benchmark for Adverse Drug Event extraction

Area of Science:

• Natural Language Processing
• Pharmacovigilance
• Machine Learning

Background:

• Active adverse event surveillance relies on extracting Adverse Drug Events (ADEs) from various data sources.
• Existing datasets and tasks primarily focus on single text types, limiting model generalizability.
• Domain generalization for ADE extraction across diverse text (e.g., scientific literature, social media) remains under-explored.

Purpose of the Study:

• To assess the current capabilities of machine learning models in extracting ADEs across different text domains.
• To develop a comprehensive benchmark for evaluating domain generalization in ADE extraction.
• To investigate the feasibility of a single ADE extraction model effective across various text types.

Main Methods:

• Construction of the MultiADE benchmark, incorporating existing datasets and the new CADECv2 dataset.
• CADECv2 extends CADEC with annotations for online posts covering a wider range of drugs.
• Annotation of the new dataset by human annotators following detailed guidelines.

Main Results:

• Trained models exhibit imperfect generalization across different text domains, hindering deployment.
• Intermediate transfer learning shows promise but requires further investigation into domain adaptation methods.
• Cost-effective methods for selecting useful training instances are crucial for improving domain adaptation.

Conclusions:

• Current ADE extraction models are not yet suitable for processing diverse text types without domain-specific adaptation.
• Domain adaptation techniques, particularly efficient instance selection, are essential for improving model performance.
• The MultiADE benchmark and CADECv2 dataset are publicly available to advance research in information extraction for pharmacovigilance.