Impact of study quality on outcome in placebo-controlled trials of homeopathy

We investigated the influence of indicators of methodological quality on study outcome in a set of 89 placebo-controlled clinical trials of homoeopathy in three different ways: (1) The results of studies meeting single criteria (explicit statement of random allocation, allocation concealment, double-blinding, completeness of follow-up) of methodological quality were compared with those of studies not meeting the criteria in univariate and multivariate analyses; (2) The results of studies scoring above and below predefined scores in two quality assessment scales were compared; (3) Primary studies were consecutively entered into a cumulative meta-analysis according to the summary scores derived from the quality assessment scales. All analyses were performed using meta-regression methods. Studies that were explicitly randomized and were double-blind as well as studies scoring above the cut-points yielded significantly less positive results than studies not meeting the criteria. In the cumulative meta-analyses, there was a trend for increasing effect sizes when more studies with lower-quality scores were added. However, there was no linear relationship between quality scores and study outcome. We conclude that in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results. Because summarizing disparate study features into a single score is problematic, meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome.