Accounting for multiplicity in the evaluation of "signals" obtained by data mining from spontaneous report adverse...

Surveillance of drug products in the marketplace continues after approval, to identify rare potential toxicities that are unlikely to have been observed in the clinical trials carried out before approval. This surveillance accumulates large numbers of spontaneous reports of adverse events along with other information in spontaneous report databases. Recently developed empirical Bayes and Bayes methods provide a way to summarize the data in these databases, including a quantitative measure of the strength of the reporting association between the drugs and the events. Determining which of the particular drug-event associations, of which there may be many tens of thousands, are real reporting associations and which random noise presents a substantial problem of multiplicity because the resources available for medical and epidemiologic followup are limited. The issues are similar to those encountered with the evaluation of microarrays, but there are important differences. This report compares the application of a standard empirical Bayes approach with micorarray-inspired methods for controlling the False Discovery Rate, and a new Bayesian method for the resolution of the multiplicity problem to a relatively small database containing about 48,000 reports. The Bayesian approach appears to have attractive diagnostic properties in addition to being easy to interpret and implement computationally.