Related Experiment Videos
Is quality control of Cochrane reviews in controversial areas sufficient?
Jan Magnus Bjordal1, Rodrigo Alvaro Brandao Lopes-Martins, Atle Klovning
1Section of Physiotherapy Science, University of Bergen and Institute of Physiotherapy, Bergen University College, Bergen, Norway. jan.bjordal@hib.no
Objectives:
Some conclusions from the Cochrane Database of Systematic Reviews (CDSR) seem volatile in areas of controversy and have recently changed conclusions. With this perspective in mind we decided to test the validity and sensitivity of negative conclusions in a systematic review of low-level laser therapy (LLLT) for osteoarthritis (OA) from the Cochrane Library was valid and robust.
Settings/Location:
None as this was a systematic review of literature.
Design:
Validity was tested against a 9-item checklist for systematic reviews. Review selections were analyzed for possible discrepancies between trial and review reports and omissions of relevant trials and data. Data from discrepancies and omissions were then entered into sensitivity and subgroup analyses.
Outcome Measures:
Continuous and dichotomous data for pain
Subjects:
Patients with osteoarthritis.
Interventions:
LLLT.
Results:
Only clinicians who had performed LLLT trials, and had negative results were invited into the review group. The review was oblivious to findings published after 1993 about physiologic mechanisms and dose response patterns for LLLT. We found 18 questionable selections that favored a negative review conclusion in 17 of 18 cases. These were largely omissions of relevant positive data and selective inclusion of negative data from trials with small, ineffective doses. When existing and omitted data from relevant trials were combined, results changed from negative to significantly positive for continuous and categorical data. Subgrouping trials by location and recommended doses taken from current guidelines, revealed a highly significant effect of LLLT for treating knee OA.
Conclusions:
The results of this CDSR was not robust and seems to be colored by questionable selections or omissions. For alternative pain therapies, it should be considered if lack of expertise on therapy mechanisms, or conflicts of interests with competing pain drug manufacturers, may bias conclusions. Review groups should recruit a balanced mix of current views and expertise and expand the use of sensitivity analyses to improve quality of CDSRs in areas of controversy.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:17Cochran's Q Test
01:05Quality Control
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
01:16Critical Region, Critical Values and Significance Level
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the test...
01:13Controls in Experiments
01:03Drug Control Governance: Regulatory Bodies and Their Impact