Related Experiment Video
Updated: Apr 11, 2026

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats
Published on: June 18, 2018
Meta-research: The art of getting it wrong
John P A Ioannidis1,2,3,4,5,6
1Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. jioannid@stanford.edu, jioannid@cc.uoi.gr.
Meta-analyses, while powerful, can yield misleading results. This research explores case studies and factors like bias and data flexibility that impact meta-analysis credibility, advocating for prospective designs.
Area of Science:
- Biostatistics
- Research Methodology
Background:
- Meta-analysis is a valuable research tool but susceptible to errors.
- Potential pitfalls include biased effect size estimation and significance-chasing.
Purpose of the Study:
- To examine case studies illustrating meta-analysis limitations.
- To identify factors influencing the credibility of meta-analyses.
- To propose prospective meta-analysis designs for improved research integrity.
Main Methods:
- Review of case studies from clinical trials and observational research.
- Discussion of meta-research principles, including bias and credibility.
- Survey of expert meta-analysts on confidence in meta-analysis results.
Main Results:
- Meta-analyses can produce incorrect and misleading conclusions.
- Factors such as data flexibility, multiplicity of analyses, and reporting practices affect credibility.
- Most expert meta-analysts doubt the precision of 95% confidence intervals.
Conclusions:
- Meta-analysis requires careful consideration of potential biases and credibility factors.
- Prospective meta-analysis designs are recommended for more reliable research outcomes.
- Integrating prospective meta-analysis into primary research can enhance overall scientific rigor.
More Related Videos
08:53Using a Classroom-Based Deese Roediger McDermott Paradigm to Assess the Effects of Imagery on False Memories
Published on: November 14, 2018
07:26The Deese-Roediger-McDermott DRM Task: A Simple Cognitive Paradigm to Investigate False Memories in the Laboratory
Published on: January 31, 2017
Related Concept Videos
Systematic Error: Methodological and Sampling Errors
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Hindsight Biases
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Metacognition
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
Errors and Mistakes in Surveying