Approaches to heterogeneity in meta-analysis

This paper reviews publications from January 1999 to March 2001 on reproductive health topics that were self-identified as meta-analysis or were indexed as meta-analysis in MEDLINE. It sought to assess whether tests of statistical heterogeneity were done, whether the results were reported, and how a finding of significance for a test of statistical heterogeneity was handled and the results interpreted. The review identified some concerns. Tests of statistical heterogeneity were not done universally even though virtually all writers on the topic emphasize their importance. Even when done, results of these tests were not universally reported. Although the consensus appears to be that heterogeneity tests are conservative for meta-analysis of studies and a probability value of 0.10 is preferred, many meta-analyses used the conventional value of 0.05 without providing a reason. The rationale for the choice of a random or fixed effects model was not generally evident. The review also provided some positive models and some recommendations for assessing, reporting and exploring heterogeneity are made considering these models and the published recommendations of experts.