Estimation of between-trial variance in sequential meta-analyses: a simulation study

Estimators for the variance between treatment effects from randomized clinical trials (RCTs) in a meta-analysis may yield divergent or even contradictory results. In a sequential meta-analysis (SMA), their properties are even more important, as they influence the point in time at which definite conclusions are drawn. In this study, we evaluated the properties of estimators of heterogeneity to be used in an SMA. We conducted an extensive simulation study with dichotomous and continuous outcome data and applied the estimators in real life examples. Bias and variance of the estimators were used as primary evaluation criteria, as well as the number of RCTs and patients from the accumulating trials needed to get stable estimates. The simulation studies showed that the well-known DerSimonian-Laird (DL) estimator largely underestimates the true value for dichotomous outcomes. The two-step DL (DL2) significantly improves this behavior. In general, the DL2 and Paule-Mandel (PM) estimators are recommended for both dichotomous and continuous outcome data for use in an SMA.