Adverse Drug Events Detection in Clinical Notes by Jointly Modeling Entities and Relations Using Neural Networks

Adverse drug events (ADEs) occur in approximately 2-5% of hospitalized patients, often resulting in poor outcomes or even death. Extraction of ADEs from clinical narratives can accelerate and automate pharmacovigilance. Using state-of-the-art deep-learning neural networks to jointly model concept and relation extraction, we achieved the highest integrated task score in the 2018 Medication and Adverse Drug Event (MADE) 1.0 challenge.

We used a combined bidirectional long short-term memory (BiLSTM) and conditional random fields (CRF) neural network to detect medical entities relevant to ADEs and a combined BiLSTM and attention network to determine relations, including the adverse drug reaction relation between medication and sign or symptom entities. Using these models, we conducted three experiments: (1) separate and sequential modeling of entities and relations; (2) joint modeling where relations between medications and sign or symptoms determined ADE and indication entities; (3) use of information from external resources such as the US FDA's adverse event database as additional input to the second method.

Joint modeling improved the overall task accuracy from 0.62 to 0.65 F measure, and the additional use of external resources improved the accuracy to 0.66 F measure. Given the gold-standard medical entity labels, the joint model plus external resources method achieved F measures of 0.83 for ADE-relevant medical entity detection and 0.87 for relation detection.

It is important to use joint modeling techniques and external resources for effectively detecting ADEs from clinical narratives in electronic health record (EHR) systems. While the extraction of entities and relations individually achieved high accuracy, the integrated task still has room for further improvement.