Trials We Cannot Trust: Investigating Their Impact on Systematic Reviews and Clinical Guidelines in Spinal Pain

Area of Science:

• Evidence-based medicine
• Clinical research integrity
• Health sciences

Background:

• Previous work identified untrustworthy clinical trials concerning cognitive-behavioral therapy (CBT) and exercise for spinal pain.
• Concerns about trial trustworthiness can propagate through the evidence synthesis process.

Purpose of the Study:

• To systematically assess the impact of untrustworthy "index trials" on subsequent systematic reviews and clinical practice guidelines (CPGs).
• To quantify how the inclusion of questionable trial data affects meta-analyses and narrative syntheses in spinal pain research.

Main Methods:

• Forward citation tracking of 8 identified untrustworthy trials.
• Systematic searching of guideline databases (Guidelines International Network, NICE) for relevant reviews and CPGs up to June 2022.
• Sensitivity analyses of meta-analyses (pain and disability outcomes) excluding index trials.
• Application of an "Impact Index" to categorize the influence of index trials.

Main Results:

• 32 systematic reviews and 10 CPGs were analyzed.
• Exclusion of index trials reduced meta-analysis effect sizes by a median of 58%.
• 85% of meta-analytic comparisons were highly impacted; 90% of narrative reviews and CPGs drew positive conclusions.

Conclusions:

• Untrustworthy trials significantly influenced the results, conclusions, and recommendations of systematic reviews and CPGs for spinal pain interventions.
• There is a critical need to enhance the appraisal of study trustworthiness in evidence synthesis and guideline development.