Impact of random assignment on study outcome: an empirical examination

Sixty research investigations published in the biomedical literature were analyzed to examine the effect of design attributes on outcome. All 60 studies included a controlled trial involving a pretest, a therapeutic intervention, and a posttest across at least two groups. Thirty of the trials used random assignment of participants to treatment or control conditions and 30 trials employed some nonrandom method of subject assignment. Trial results were aggregated and evaluated by comparing effect sizes for the primary statistical test of the hypothesis. Data analysis revealed that the trial results, as measured by effect size, did not vary across therapeutic trials using random assignment versus those using nonrandom allocation of subjects. The impact of design attributes in the interpretation of multiple clinical trials addressing a similar research question is examined. The argument is made that various design attributes frequently associated with methodological quality should be considered as important moderator variables and their influence on trial outcome should not be assumed a priori but rather examined empirically.