Adjusting for publication bias in the presence of heterogeneity

It is known that the existence of publication bias can influence the conclusions of a meta-analysis. Some methods have been developed to deal with publication bias, but issues remain. One particular method called 'trim and fill' is designed to adjust for publication bias. The method, which is intuitively appealing and comprehensible by non-statisticians, is based on a simple and popular graphical tool called the funnel plot. We present a simulation study designed to evaluate the behaviour of this method. Our results indicate that when the studies are heterogeneous (that is, when they estimate different effects), trim and fill may inappropriately adjust for publication bias where none exists. We found that trim and fill may spuriously adjust for non-existent bias if (i) the variability among studies causes some precisely estimated studies to have effects far from the global mean or (ii) an inverse relationship between treatment efficacy and sample size is introduced by the studies' a priori power calculations. The results suggest that the funnel plot itself is inappropriate for heterogeneous meta-analyses. Selection modelling is an alternative method warranting further study. It performed better than trim and fill in our simulations, although its frequency of convergence varied, depending on the simulation parameters.