Effects of directionality of significance tests on the bias of accessible effect sizes

The proportion of studies that use one-tailed statistical significance tests (pi) in a population of studies targeted by a meta-analysis can affect the bias of the sample effect sizes (sample ESs, or ds) that are accessible to the meta-analyst. H. C. Kraemer, C. Gardner, J. O. Brooks, and J. A. Yesavage (1998) found that, assuming pi = 1.0, for small studies (small Ns) the overestimation bias was large for small population ESs (delta < or = 0.2) and reached a maximum for the smallest population ES (viz., delta = 0). The present article shows (with a minor modification of H. C. Kraemer et al.'s model) that when pi = 0, the small-N bias of accessible sample ESs is relatively small for delta < or = 0.2, and a minimum (in fact, nonexistent) for delta = 0. Implications are discussed for interpretations of meta-analyses of (a) therapy efficacy and therapy effectiveness studies, (b) comparative outcome studies, and (c) studies targeting small but important population ESs.