Intelligent Automation Improved Efficiency in Pharmacovigilance Safety Signal Assessment

Pharmacovigilance is the science involving the detection, assessment, understanding, and prevention of adverse events associated with drugs, biologics, or medical devices. In pharmacovigilance, information that suggests a new potential causal relationship between an intervention and an adverse event is called a safety signal. Following their detection, safety signals are assessed via a comprehensive, structured analysis to more fully elucidate whether a correlation exists. This key process often requires the manual assessment of many individual case safety report (ICSR) narratives to extract meaningful information in a labor-intensive, time-consuming, and variability-prone manner. In this retrospective feasibility study, we describe the potential utility of an intelligent automation system leveraging the GPT-4o large language model to automate the extraction of case elements of interest from a series of ICSR narratives while maintaining human expert oversight. Our proprietary platform allowed users to extract the presence or absence of risk factors and responses to dechallenge and rechallenge via instructional prompts built on a common template structure. Case elements for five historical signal assessments were selected based on need for and feasibility of artificial intelligence extraction. Performance ranged from F1 = 0.444 to 1.000 for risk factors and from F1 = 0.429 to 0.909 for responses to dechallenge and rechallenge. Even when considering the need to verify GPT-4o outputs for accuracy, potential time savings were identified. To the best of our knowledge, our results are the first to demonstrate an intelligent automation platform that may streamline signal management workflows using a machine-first, human-verified operational workflow while maintaining regulatory compliance.