Unpublished trials affected evidence synthesis substantially when estimating medication harms in children

To investigate the impact of unpublished trials on the pooled estimates for medication harms.

Cochrane reviews published up to February 2023 that evaluated drug interventions and adverse events in children were collected, along with the metadata from rm5 files. The prevalence of unpublished trials among the meta-analyses was summarized, and then the meta-analytic effects were replicated by removing these trials. Association between percentage of unpublished trials and the impact were examined via logistic regression, measured by odds ratio (OR) and its 95% confidence interval.

Among 740 eligible meta-analyses, 14.9% included unpublished trials. Excluding these trials changed effect direction in 26.4% of meta-analyses, statistical significance in 28.2%, and magnitude by >50% in 8.2%. Subgroup analyses showed that the impact was more pronounced in smaller meta-analyses, regardless of the effect direction (less than five studies vs those with 10 or more studies: 42.9% vs 4.8%) or the statistical significance (46.0% vs 4.8%). Higher percentage of unpublished trials showed larger impact, regardless of the direction or significance (eg, 50% vs 20%, significance: odds ratio = 11.5, 95% confidence interval: 4.7-28.6, P< .001).

Even a small number of eligible unpublished trials can substantially alter pooled effect estimates, particularly in drug safety research. Systematic reviews should include a comprehensive search for unpublished harms data from trials.