The quality of trials in operative surgery

This study aimed to assess the reported quality of trials in operative surgery.

Randomized controlled trials (RCTs) in operative surgery have previously been criticized for using weak methodology despite no evidence to suggest their quality is any different from nonsurgical trials.

All surgical RCTs published in the British Medical Journal, the Journal of the American Medical Association, The Lancet, and the New England Journal of Medicine between 1998 and 2004 were identified. The adequacy of the reported methodology used to perform the randomization, power calculation, and recruitment was assessed for each trial using predefined criteria. The results from the surgical trials were compared with a randomly selected control group of nonsurgical RCTs, which were matched for journal and year of publication.

Sixty-six surgical RCTs were identified. Adequate reporting of randomization sequence generation was seen in 42% (n = 28) of surgical trials and 30% (n = 20) of nonsurgical trials, and adequate allocation concealment was recorded in 46% (n = 30) and 47% (n = 31), respectively. When combining these 2 interrelated steps of randomization, only 26% (n = 17) of surgical trials and 23% (n = 15) of nonsurgical trials reported both adequately. Adequate recruitment was recorded in 52% (n = 33 of 63) surgical and 55% (n = 33 of 60) nonsurgical trials, with approximately a quarter (n = 17 and n = 16, respectively) of the trials in both the surgical and nonsurgical categories reporting an adequate power calculation.

There was no evidence that the reported quality of surgical trials was different to nonsurgical trials. However, approximately half or less of all the trials reviewed reported adequate methodology.