Comparison of laboratory threshold criteria in drug-induced liver injury detection algorithms for use in pharmacovigilance

For the purpose of pharmacovigilance, we sought to determine the best performing laboratory threshold criteria to detect drug-induced liver injury (DILI) in the electronic medical records (EMR).

We compared three commonly used liver chemistry criteria from the DILI expert working group (DEWG), DILI network (DILIN), and Council for International Organizations of Medical Sciences (CIOMS), based on hospital EMR for years 2010 and 2011 (42 176 admissions), using independent medical record review. The performance characteristics were compared in terms of sensitivity, specificity, positive predictive value (PPV), negative predictive value, accuracy, F-measure, and area under the receiver operating characteristic curve (AUROC).

DEWG had the highest PPV (5.5%, 95% CI: 4.1%-7.2%), specificity (97.0%, 95% CI: 96.8%-97.2%), accuracy (96.8%, 95% CI: 96.6%-97.0%) and F-measure (0.099). CIOMS had the highest sensitivity (74.0%, 95% CI: 64.3%-82.3%) and AUROC (85.2%, 95% CI: 80.8%-89.7%). Besides the laboratory criteria, including additional keywords in the classification algorithm improved the PPV and F-measure to a maximum of 29.0% (95% CI: 22.3%-36.5%) and 0.379, respectively.

More stringent criteria (DEWG and DILIN) performed better in terms of PPV, specificity, accuracy and F-measure. CIOMS performed better in terms of sensitivity. An algorithm with high sensitivity is useful in pharmacovigilance for detecting rare events and to avoid missing cases. Requiring at least two abnormal liver chemistries during hospitalization and text-word searching in the discharge summaries decreased false positives without loss in sensitivity.