A retrospective quality analysis of 102 randomized trials in four leading urological journals from 1984-1989

The objective of the present study was to analyse critically the quality of the reporting in 102 randomized trials from four leading urological journals from 1984 to 1989 on the basis of an evaluation system we have developed. This comprises 21 principal parameters selected in terms of their significance for the validity of the studies. These parameters were evaluated by two readers independently of each other as to whether they were specified, not specified, could not be evaluated or were not applicable. The study score of each paper resulted from the sum of all specified criteria. In the 102 studies, out of 21 criteria 69.1% and 69.8% (investigators A and B, respectively) were reported; 29.8% and 29.4%, respectively were not reported, 0.4% and 0.1%, respectively, could not be evaluated and 0.7% did not apply. Such important principal parameters as the sample size (6.9% and 7.8%, respectively), statistical power (11.8%), method of randomization (22.5% and 23.5%, respectively), patient blinding (30.4%), investigator blinding (33.3%), loss to follow-up (34.8% and 35.3%, respectively) and rate of discontinuation (36.0% and 37.7%, respectively) were mentioned least often. The study score of all investigations ranged from 20.5 (97.6%) to 9.0 (42.9%) points. Most (60/59% and 62/61%, respectively) attained values between 16 (76.2%) and 13 (61.9%). Accordingly, randomized trials in urological journals show similar deficits to those in internal medicine, surgery and intensive care medicine. A particular problem is that they concern the most important techniques for systematic reduction of inadvertent errors (bias), and thus doubt is cast upon the hardcore of controlled studies. If it is possible for many authors to mention individual criteria completely, this should also apply (and in particular) to the most critical parameters. In our opinion, the 21 criteria selected for an evaluation system constitute a practical compromise between the 3 and 38 criteria alternatively suggested by other authors. Moreover, use of a comprehensive check list should be the precondition for acceptance of papers for publication.