Related Experiment Videos
Discrepancies among megatrials.
T A Furukawa1, D L Streiner, S Hori
1Department of Psychiatry, Nagoya City University Medical School, Mizuho-cho, Mizuho-ku, 467-8601, Nagoya, Japan. furukawa@med.nagoya-cu.ac.jp
Journal of Clinical Epidemiology
|January 9, 2001
Summary
Comparing large clinical trials (megatrials) reveals significant variability, questioning their reliability as a gold standard. Careful analysis is crucial for both megatrials and meta-analyses to ensure accurate scientific conclusions.
Area of Science:
- Clinical Epidemiology
- Biostatistics
- Evidence-Based Medicine
Background:
- Meta-analyses are often validated against megatrials, considered a gold standard in medical research.
- Recent studies suggest potential inconsistencies even when comparing megatrials to each other.
Purpose of the Study:
- To assess the reliability of megatrials by comparing results from independent megatrials on identical research questions.
- To evaluate the consistency of statistical conclusions derived from large-scale clinical studies.
Main Methods:
- Identified pairs of megatrials (≥1000 subjects) from the Cochrane Library and a specific publication (LeLorier et al., 1997).
- Compared patient-intervention-outcome combinations across identified megatrials.
- Analyzed the statistical agreement of reported outcomes using odds ratios, weighted mean differences, and kappa statistics.
Main Results:
- In 289 Cochrane Library megatrials pairs, 73% showed statistically compatible results; quadratic weighted kappa was 0.40.
- In 133 comparisons from LeLorier et al., 73% reported compatible odds ratios; kappa was 0.33.
- Agreement between independent megatrials was comparable to agreement between meta-analyses and megatrials.
Conclusions:
- The reliability of megatrials as a definitive gold standard is questionable due to observed variability.
- Rigorous critical appraisal is essential for all study types, including meta-analyses and megatrials, to ensure scientific validity.