Mapping Strategies to Assess and Increase the Validity of Published Disproportionality Signals: A Meta-Research Study

Disproportionality analysis is traditionally used in spontaneous reporting systems to generate working hypotheses about potential adverse drug reactions: the so-called disproportionality signals. We aim to map the methods used by researchers to assess and increase the validity of their published disproportionality signals.

From a systematic literature search of published disproportionality analyses up until 1 January 2020, we randomly selected and analyzed 100 studies. We considered five domains: (1) rationale for the study, (2) design of disproportionality analyses, (3) case-by-case assessment, (4) use of complementary data sources, and (5) contextualization of the results within existing evidence.

Among the articles, multiple strategies were adopted to assess and enhance the results validity. The rationale, in 95 articles, was explicitly referred to the accrued evidence, mostly observational data (n = 46) and regulatory documents (n = 45). A statistical adjustment was performed in 34 studies, and specific strategies to correct for biases were implemented in 33 studies. A case-by-case assessment was complementarily performed in 35 studies, most often by investigating temporal plausibility (n = 26). Complementary data sources were used in 25 articles. In 78 articles, results were contextualized using accrued evidence from the literature and regulatory documents, the most important sources being observational (n = 45), other disproportionalities (n = 37), and case reports (n = 36).

This meta-research study highlighted the heterogeneity in methods and strategies used by researchers to assess the validity of disproportionality signals. Mapping these strategies is a first step towards testing their utility in different scenarios and developing guidelines for designing future disproportionality analysis.