A comparison of heterogeneity variance estimators in combining results of studies

For random effects meta-analysis, seven different estimators of the heterogeneity variance are compared and assessed using a simulation study. The seven estimators are the variance component type estimator (VC), the method of moments estimator (MM), the maximum likelihood estimator (ML), the restricted maximum likelihood estimator (REML), the empirical Bayes estimator (EB), the model error variance type estimator (MV), and a variation of the MV estimator (MVvc). The performance of the estimators is compared in terms of both bias and mean squared error, using Monte Carlo simulation. The results show that the REML and especially the ML and MM estimators are not accurate, having large biases unless the true heterogeneity variance is small. The VC estimator tends to overestimate the heterogeneity variance in general, but is quite accurate when the number of studies is large. The MV estimator is not a good estimator when the heterogeneity variance is small to moderate, but it is reasonably accurate when the heterogeneity variance is large. The MVvc estimator is an improved estimator compared to the MV estimator, especially for small to moderate values of the heterogeneity variance. The two estimators MVvc and EB are found to be the most accurate in general, particularly when the heterogeneity variance is moderate to large.