Methods of randomized controlled clinical trials in health services research

The randomized controlled clinical trial is an increasingly used method in health services research. Analysis of methodology is needed to accelerate practical implementation of trial results, select trials for meta-analysis, and improve trial quality in health services research. The objectives of this study are to explore the methodology of health services research trials, create and validate a streamlined quality evaluation tool, and identify frequent quality defects and confounding effects on quality. The authors developed a quality questionnaire that contained 20 evaluation criteria for health services research trials. One hundred one trials from the Columbia Registry of Controlled Clinical Trials were evaluated using the new quality tool. The overall agreement between independent reviewers, Cohen's kappa, was 0.94 (+/- 0.01). Of a possible score of 100, the trials received an average score of 54.8 (+/- 12.5). Five evaluation criteria indicated significant quality deficiencies (sample size, description of case selection, data on possible adverse effects, analysis of secondary effect variables, and retrospective analysis). The quality of study characteristics was significantly weaker than the quality of reporting characteristics (P < 0.001). The total average scores of Medline-indexed journals were better than the non-Medline-indexed journals (P < 0.001). There was a positive correlation between the overall quality and year of publication (R = 0.21, P < 0.05). The authors conclude that the new quality evaluation tool leads to replicable results and there is an urgent need to improve several study characteristics of clinical trials. In comparison to drug trials, site selection, randomization, and blinding often require different approaches in health services research.