Related Experiment Video
Updated: Jul 18, 2026
Meta-Analysis of the Effectiveness and Safety of Shugan Jieyu Capsules for the Treatment of Insomnia
Published on: February 17, 2023
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Peter Herbison1, Jean Hay-Smith, William J Gillespie
1Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, PO Box 913, Dunedin, New Zealand. peter.herbison@otago.ac.nz
Objective:
To find if a particular quality score was better than others at validly scoring the quality of randomized controlled trials, both by examining the consistency of dividing studies into high and low quality and using a large study as a reference standard.
Study Design And Setting:
Observational study of meta-analyses from the Cochrane Library. These had to have binary outcomes that included more than 10 studies, one or more of which randomized more than 500 people into each group.
Results:
Eighteen systematic reviews, with 65 meta-analyses using binary outcomes, were included and the included trials were scored for 43 different quality scores. None of these scores was better at dividing the studies in to low and high quality, and none of the scores was better over the 65 meta-analyses in making the result closer to the reference standard.
Conclusion:
None of the quality scores found appeared to measure quality validly. It is a mistake to assign meaning to the result of a quality score.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:28Methods of Medium Optimization
01:22Pharmaceutical Alternatives: Stability-Related Therapeutic Nonequivalence