Performance of Between-study Heterogeneity Measures in the Cochrane Library

The growth in comparative effectiveness research and evidence-based medicine has increased attention to systematic reviews and meta-analyses. Meta-analysis synthesizes and contrasts evidence from multiple independent studies to improve statistical efficiency and reduce bias. Assessing heterogeneity is critical for performing a meta-analysis and interpreting results. As a widely used heterogeneity measure, the I statistic quantifies the proportion of total variation across studies that is caused by real differences in effect size. The presence of outlying studies can seriously exaggerate the I statistic. Two alternative heterogeneity measures, the (Equation is included in full-text article.)and (Equation is included in full-text article.)have been recently proposed to reduce the impact of outlying studies. To evaluate these measures' performance empirically, we applied them to 20,599 meta-analyses in the Cochrane Library. We found that the (Equation is included in full-text article.)and (Equation is included in full-text article.)have strong agreement with the I, while they are more robust than the I when outlying studies appear.