A meta-epidemiological study found lack of transparency and poor reporting of disproportionality analyses for signal...

To review and appraise methods and reporting characteristics of pharmacovigilance disproportionality analyses.

We randomly selected 100 disproportionality analyses indexed in Medline found during a systematic literature search. We then extracted and synthetized methodological and reporting characteristics using seven key items: (1) title transparency; (2) protocol pre-registration; (3) date of data extraction and analysis; (4) outcome, population, exposure and comparator definitions; (5) adjustment and stratification of results; (6) method and threshold for signal detection; (7) secondary and sensitivity analyses.

We found that methods used to generate disproportionality signals were extremely heterogeneous; there were nearly as many unique analyses as studies. The authors used various populations, methods, signal detection thresholds, adjustment or stratification variables, generally without justification for their choice or pre-specification in protocols. Moreover, 78% of studies failed to report methods for case, adverse drug reactions or comparator selection and 32 studies did not define the threshold for signal generation.

Our survey raises major concerns regarding all aspects of disproportionality analyses that could lead to misleading results and generate unjustified alarms. We advocate for a strong and transparent rationale for variable selection, choice of population and comparators pre-specified in a protocol and assessed by sensitivity analyses.