Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research

Random sampling and random assignment (randomization) are some of the most popular methods of equating contrasted groups on pre-existing nuisance variables. However, the small samples typically used in psychotherapy outcome studies raise some questions about the extent to which these methods eliminate the pretreatment nonequivalence of groups in this area of research. This article identifies conditions under which equivalence is likely (and unlikely) to be attained with simple random sampling and randomization in psychotherapy efficacy studies of the kind examined in recent meta-analyses. Some consequences of nonequivalence are viewed as manifestations of Simpson's paradox. Misinterpretations of estimates of the relative efficacy of treatments are expected in view of belief in the law of small numbers. The minimum sample sizes needed to protect against nonequivalence are compared with those needed to satisfy several other criteria.