A new measure of between-studies heterogeneity in meta-analysis

Assessing the magnitude of heterogeneity in a meta-analysis is important for determining the appropriateness of combining results. The most popular measure of heterogeneity, I(2) , was derived under an assumption of homogeneity of the within-study variances, which is almost never true, and the alternative estimator, R^I, uses the harmonic mean to estimate the average of the within-study variances, which may also lead to bias. This paper thus presents a new measure for quantifying the extent to which the variance of the pooled random-effects estimator is due to between-studies variation, R^b, that overcomes the limitations of the previous approach. We show that this measure estimates the expected value of the proportion of total variance due to between-studies variation and we present its point and interval estimators. The performance of all three heterogeneity measures is evaluated in an extensive simulation study. A negative bias for R^b was observed when the number of studies was very small and became negligible as the number of studies increased, while R^I and I(2) showed a tendency to overestimate the impact of heterogeneity. The coverage of confidence intervals based upon R^b was good across different simulation scenarios but was substantially lower for R^I and I(2) , especially for high values of heterogeneity and when a large number of studies were included in the meta-analysis. The proposed measure is implemented in a user-friendly function available for routine use in r and sas. R^b will be useful in quantifying the magnitude of heterogeneity in meta-analysis and should supplement the p-value for the test of heterogeneity obtained from the Q test. Copyright © 2016 John Wiley & Sons, Ltd.