Pharmacovigilance data mining with methods based on false discovery rates: a comparative simulation study

The early detection of adverse reactions caused by drugs that are already on the market is the prime concern of pharmacovigilance efforts; the methods in use for postmarketing surveillance are aimed at detecting signals pointing to potential safety concerns, on the basis of reports from health-care providers and from information available in various databases. Signal detection methods based on the estimation of false discovery rate (FDR) have recently been proposed. They address the limitation of arbitrary detection thresholds of the automatic methods in current use, including those last updated by the US Food and Drug Administration and the World Health Organization's Uppsala Monitoring Centre. We used two simulation procedures to compare the false-positive performances for three current methods: the reporting odds ratio (ROR), the information component (IC), the gamma Poisson shrinkage (GPS), and also for two FDR-based methods derived from the GPS model and Fisher's test. Large differences in FDR rates were associated with the signal-detection methods currently in use. These differences ranged from 0.01 to 12% in an analysis that was restricted to signals with at least three reports. The numbers of signals generated were also highly variable. Among fixed-size lists of signals, the FDR was lowered when the FDR-based approaches were used. Overall, the outcomes in both simulation studies suggest that improvement in effectiveness can be expected from use of the FDR-based GPS method.