Deficits and remedy of the standard random effects methods in meta-analysis

The random effects model is often used in meta-analyses. A corresponding significance test based on a normal approximation has been established. Its type I error is derived in this article by theoretical considerations and computer simulations. The test can be conservative as well as unacceptably anti-conservative. The anti-conservatism increases with the increasing number of patients and the decreasing number of studies. A modification is proposed, which keeps the nominal level asymptotically as the number of patients approaches infinity. Simulations show that the modified test is often conservative, but its conservatism is small in those situations where the standard test is highly anti-conservative.