Time-to-signal comparison for drug safety data-mining algorithms vs. traditional signaling criteria

Data mining may improve identification of signals, but its incremental utility is in question. The objective of this study was to compare associations highlighted by data mining vs. those highlighted through the use of traditional decision rules. In the case of 29 drugs, we used US Food and Drug Administration (FDA) Adverse Event Reporting System (AERS) data to compare three data-mining algorithms (DMAs) with two traditional decision rules: (i) N >or= 3 reports for a designated medical event (DME) and (ii) any event comprising >2% of reports in relation to a drug. Data-mining methods produced 101-324 signals vs. 1,051 for the N >or= 3 rule but yielded a higher proportion of signals having publication support. For the 2% rule, the fraction of signals having publication support was similar to that associated with data mining. Data-mining signals lagged N >or= 3 signaling by 1.5-11.0 months. It may therefore be concluded that data mining identifies fewer signals than the "N >or= 3 DME" rule. The signals appear later with data mining but are more often supported by publications. In the case of the 2% rule, no such difference in publication support was observed.