Estimating the proportion of studies missing for meta-analysis due to publication bias

A potential problem with meta-analysis concerns missing studies due to publication bias. This paper focuses on one subtype of publication bias, namely selection bias (studies with unfavorable outcomes tend to be suppressed), wherein the main interest is in determining the proportion of unpublished studies.

As in the well-known trim and fill method, the key assumption is that studies with quantitative outcomes extremely unfavorable for the treatment are not published. Along with the assumption of a normal distribution for the complete set of published and unpublished studies, the proportion of unpublished studies is estimated by the degree of truncation from a left-truncated normal distribution. In addition, the mean and variance of this distribution are obtained in order to provide useful information regarding the mean effect of a treatment vs. a control, and the variance of this effect when controlling for truncation. The degree of truncation can be considered under two hypotheses: the true mean equals the estimated mean or the true mean is equal to zero.

The uncorrected degree of truncation was found to be overestimated, but this bias was reduced when correcting for chance truncation. To incorporate additional information, weighted analysis was proposed. Instead of unweighted mean and variance of the published outcomes, their weighted analogues were used for analysis, with the weights expressing varying credibility across studies entering the meta-analysis. One hypothetical and three empirical examples illustrated the approach.

The new method is very simple and its results are comparable to other meta-analytic methods. However, in contrast to existing methods, it can be applied to the study-specific outcomes alone.