Related Experiment Video
Updated: Jan 2, 2026
Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
Discrepancies in meta-analyses answering the same clinical question were hard to explain: a meta-epidemiological
1Department of Pediatrics I, University Medical Center Schleswig-Holstein, Campus Kiel, Arnold-Heller-Straße 3, Kiel 24105, Germany; Christian-Albrechts-University of Kiel, Institute of Sports Science, Olshausenstraße 74, 24118 Kiel, Germany.
Objectives:
To systematically explore the methodological factors underpinning discrepancies in the pooled effect estimates from Cochrane reviews (CRs) and non-Cochrane reviews (NCRs) systematic reviews, answering the same clinical question.
Study Design And Setting:
Quantitative and qualitative analysis of concordance in effect estimates between meta-analyses from CR and NCR matched on population, intervention, condition, and outcome.
Results:
We identified 24 matched meta-analyses from 24 CR to 20 NCR reviews (545 randomized controlled trials [RCTs]). Compared to their CR matched-pair, pooled effects from NCR were the same in only one pair, were on average 0.12 log units (13%) higher (P = 0.012), and had a greater than twofold larger effect size in four matched-pairs. Two-thirds of CR (15/24, 70.8%) and 0/20 (0%) NCR were rated to have moderate to high confidence in their results (AMSTAR 2). Differences in pre-defined methods, including search strategy, eligibility criteria, and performance of dual screening, could explain mismatches in included studies. Disagreements in the interpretation of eligibility criteria were identified as reasons underpinning discrepant findings in 14 pairs. 23/24 meta-analyses included at least one study of its match. Only two pairs agreed on the numerical data presented for the same studies. An assessment of 50% of discrepant studies (n = 45) showed that reasons for differences in extracted data could be identified in 15 studies.
Conclusion:
On average, meta-analyses from NCR reported higher effect estimates compared with meta-analyses from CR answering the same clinical question. Methodological and author judgments and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported effect estimates. The potential impacts on health care policy and clinical practice are far-reaching but still remain unknown. Reinforcing awareness and scrutiny of application of reporting guidelines and improvements in protocol registration are needed.
More Related Videos
06:26Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
04:34Meta-Analysis of the Effectiveness and Safety of Shugan Jieyu Capsules for the Treatment of Insomnia
Published on: February 17, 2023
Related Concept Videos
01:29Bias in Epidemiological Studies
01:27Confounding in Epidemiological Studies
01:26Introduction to Epidemiology
01:20Study Designs in Epidemiology
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
01:25Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...