Quality evaluation of controlled clinical information service trials

Randomized controlled clinical trials are increasingly accepted as tools of computer technology assessment and, therefore, quality evaluation of trials has great theoretical and practical significance. The purpose of this study was to assist the design of evaluation studies and synthesis of published results by developing and validating an easy-to-use quality scoring method. The development of the new scoring system was based on the available quality evaluation methods and the analysis of 19 trial reports registered in the Columbia Registry of Controlled Clinical Information Service Trials. First critical aspects and afterwards the levels of quality were defined. In spite of the fact that all quality requirements were met by some trials, the average overall quality score was 52.6 (+/- 8.7) per cent. The minimum score was 37 and the maximum was 72 per cent. Data collection and site/sample definition were better in the good quality trials, but improvement in statistical analysis was erratic. The quality scoring method was validated by using another sample of 20 registered trials. While the number of published controlled clinical trials is increasing in medical informatics, the analysis was unable to demonstrate a significant positive correlation between the quality and year of publication.