Randomized versus historical controls for clinical trials

To compare the use of randomized controls (RCTs) and historical controls (HCTs) for clinical trials, we searched the literature for therapies studied by both methods. We found six therapies for which 50 RCTs and 56 HCTs were reported. Forty-four of 56 HCTs (79 percent) found the therapy better than the control regimen, but only 10 of 50 RCTs (20 percent) agreed. For each therapy, the treated patients in RCTs and HCTs of the same therapy was largely due to differences in outcome for the control groups, with HCT control patients generally doing worse than the RCT control groups. Adjustment of the outcomes of the HCTs for prognostic factors, when possible, did not appreciably change the results. The data suggest that biases in patient selection may irretrievably weight the outcome of HCts in favor of new therapies. RCTs may miss clinically important benefits because of inadequate attention to sample size. The predictive value of each might be improved by reconsidering the use of p less than 0.05 as the significance level for all types of clinical trials, and by the use of confidence intervals around estimates of treatment effects.