Impact of approximating or ignoring within-study covariances in multivariate meta-analyses

Multivariate meta-analyses are used to derive summary estimates of treatment effects for two or more outcomes from a joint model. In addition to treatment effects, these models also quantify the correlations between outcomes across studies. To be fully specified, the model requires an estimate of the covariance or correlations between outcomes observed in each study. These are rarely available in published reports, so that analysts must either approximate these or ignore correlations between effect estimates from the same studies. We examined the impact of errors in approximating within-study covariances on the parameters of multivariate models in a simulation study. We found that treatment effect and heterogeneity estimates were not strongly affected by inaccurate approximations, but estimates of the correlation between outcomes were sometimes highly biased. The potential for error is greatest when the covariance between outcomes within- and between-studies are of comparable scale.