Evaluating methods for testing publication bias in meta-analyses with confounding covariates: A simulation study

Selective publication of favorable results can bias meta-analytic estimates and threaten the validity of meta-analyses. Many methods that detect and adjust for publication bias perform poorly under substantial between-study heterogeneity. Heterogeneity complicates publication bias assessment because study characteristics may be associated with both effect size estimates and standard errors. These characteristics can induce an effect-precision association that regression-based diagnostics may misattribute to selective publication when such characteristics are omitted. However, it remains unclear how best to incorporate these confounding characteristics across methods and how their performance varies with confounding level, selective publication, residual heterogeneity, and the number of studies. To address this gap, I conduct an extensive Monte Carlo simulation evaluating Egger-type regression tests, residual-based tests, and a likelihood-ratio test of the three-parameter selection model (3PSM) under the one-step p value selection mechanism. Results show that when confounding covariates were ignored, Egger-type regression tests and residual analyses exhibited inflated Type I error rates, whereas the 3PSM likelihood-ratio test maintained near-nominal, albeit slightly inflated, Type I error rates across conditions. When confounding covariates were incorporated, Egger-type tests had well-calibrated Type I error rates, residual analyses based on residualizing effect sizes alone tended to be conservative, and the 3PSM test showed slight inflation. Nevertheless, 3PSM consistently yielded higher power than other methods. Overall, the findings suggest that Egger-type regression tests and 3PSM can be used as complementary analyses for assessing selective publication when confounding covariates are present.