NLP in Support of Pharmacovigilance: QUality Adverse Drug Reaction AcTIve Control (QUADRATIC)

Adverse drug reactions (ADRs) are a major cause of morbidity, hospital admissions, and in-hospital mortality, yet remain incompletely captured by post-marketing pharmacovigilance, which suffers from underreporting. Electronic health records (EHRs) contain clinical narratives that can reveal otherwise unreported ADRs. Natural language processing (NLP) offers a scalable means to extract structured information from clinical narratives, supporting ADR detection and assessment. We conducted a retrospective cross-sectional study within a multisite hospital network in Southern Switzerland to develop and evaluate NLP systems for ADR detection and information extraction from electronic discharge summaries. ADR classification models were trained on 400 discharge summaries and compared across multiple machine learning and vectorization strategies against a regular expression (regex) system. Drug and clinical event extraction were evaluated using 100 manually annotated summaries, benchmarking a dictionary-based approach against a two-step deep learning (DL) pipeline integrating transformer-based named entity recognition (NER) with a pharmacovigilance-oriented contextual relevance classifier. Performance was evaluated using standard metrics and a custom top-k ranking metric aligned with pharmacovigilance experts' daily capacity for reviewing positive cases to confirm the presence of ADRs. Logistic regression with Bag-of-Words achieved the best overall performance, combining high precision and effective case ranking. In a simulated deployment, this model identified nearly twice as many discharge summaries containing confirmed ADRs than as regex system. The two-step DL pipeline outperformed the dictionary-based approach for drug and clinical event recognition and accurately classified them according to pharmacovigilance purposes. These results demonstrate that NLP-based analysis of real-world clinical narratives can enhance pharmacovigilance while maintaining a manageable expert workload.