Impact of stratification on adverse drug reaction surveillance

Automated screening for excessive adverse drug reaction (ADR) reporting rates has proven useful as a tool to direct clinical review in large-scale drug safety signal detection. Some measures of disproportionality can be adjusted to eliminate any undue influence on the ADR reporting rate of covariates, such as patient age or country of origin, by using a weighted average of stratum-specific measures of disproportionality. Arguments have been made in favour of routine adjustment for a set of common potential confounders using stratification. The aim of this paper is to investigate the impact of using adjusted observed-to-expected ratios, as implemented for the Empirical Bayes Geometric Mean (EBGM) and the information component (IC) measures of disproportionality, for first-pass analysis of the WHO database.

A simulation study was carried out to investigate the impact of simultaneous adjustment for several potential confounders based on stratification. Comparison between crude and adjusted observed-to-expected ratios were made based on random allocation of reports to a set of strata with a realistic distribution of stratum sizes. In a separate study, differences between the crude IC value and IC values adjusted for (combinations of) patient sex, age group, reporting quarter and country of origin, with respect to their concordance with a literature comparison were analysed. Comparison was made to the impact on signal detection performance of a triage criterion requiring reports from at least two countries before a drug-ADR pair was highlighted for clinical review.

The simulation study demonstrated a clear tendency of the adjusted observed-to-expected ratio to spurious (and considerable) underestimation relative to the crude one, in the presence of any very small strata in a stratified database. With carefully implemented stratification that did not yield any very small strata, this tendency could be avoided. Routine adjustment for potential confounders improved signal detection performance relative to the literature comparison, but the magnitude of the improvement was modest. The improvement from the triage criterion was more considerable.

Our results indicate that first-pass screening based on observed-to-expected ratios adjusted with stratification may lead to missed signals in ADR surveillance, unless very small strata are avoided. In addition, the improvement in signal detection performance due to routine adjustment for a set of common confounders appears to be smaller than previously assumed. Other approaches to improving signal detection performance such as the development of refined triage criteria may be more promising areas for future research.