Efficacy of and errors in randomized multicenter trials. A review of 230 clinical trials

An attempt was made to study the efficacy in and one of the errors of randomized multicenter clinical trials. Efficacy was defined for the present purpose as a statistically significant clinical difference between at least 2 of the treatment arms studied. The error studied was defined here as non-excluded protocol deviations found during a systematic record review. The nature (multicenter randomized trial or un-controlled pilot study) of 8 so-called "breakthrough" studies which seemed particularly important for future cancer treatment was also established. The number of yearly randomized trials seems to have increased by a factor of 10 between 1969 and 1981, as judged by the abstracts published by the American Society of Clinical Oncology. 23% yielded statistically significant results, which is 18% over the 5% expected chance significances. Of 174 papers and abstracts published by one group, only 18 were reviewable original studies. The corresponding figures for another group was 6 of 22. This explains why only 8 scientifically new statistical significances were found in one series of 76 original articles. The mean dose of cytostatics actually given to 192 breast cancer patients varied, as judged by a record review, between 71 and 93% of the protocol doses. These figures agree with the 15-30% of nonexcluded protocol deviations found previously. Of the 8 "breakthrough" studies, 7 were single center pilot studies without randomized controls, and only 1 was a randomized multicenter study. It is suggested that more relevant questions can be asked in multicenter randomized clinical trials if they are based on promising single center pilot results, and that record reviews should be performed so that protocol deviations can regularly be reported.