Does the non-randomized controlled study have a place in the systematic review? A pilot study.

Area of Science:

• Evidence-based medicine
• Systematic review methodology
• Clinical trial design

Background:

• Systematic reviews traditionally prioritize randomized controlled trials (RCTs), often excluding valuable non-randomized studies.
• This exclusion can lead to indeterminate reviews, especially in fields like forensic mental health where RCTs are challenging.
• Expanding inclusion criteria to include high-quality non-randomized studies could enhance systematic review informativeness.

Purpose of the Study:

• To investigate if well-designed, controlled non-randomized studies can serve as reliable alternatives to RCTs in systematic reviews.
• To compare outcomes between RCTs and non-randomized studies, and between high- and low-quality studies within existing reviews.

Main Methods:

• Pilot study analyzing two Cochrane Schizophrenia Group reviews.
• Comparison of outcomes between included RCTs and excluded non-randomized studies.
• Assessment of study quality using a standardized checklist to compare high- and low-quality studies.

Main Results:

• In one review, effect sizes were comparable between randomized and non-randomized studies, irrespective of quality.
• In the second review, RCTs showed a lower treatment effect; high-quality studies consistently demonstrated smaller effect sizes than low-quality studies.

Conclusions:

• High-quality non-randomized controlled studies can provide outcomes comparable to RCTs.
• Overall study quality appears to influence treatment effect size more than randomization alone.
• Recommendations for future research are proposed to address unresolved issues in evidence synthesis.