Related Experiment Video
Updated: Oct 17, 2025
06:26
Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
74.3K
Retrospective median power, false positive meta-analysis and large-scale replication
T D Stanley1, Hristos Doucouliagos2, John P A Ioannidis3,4
1Deakin Lab for the Meta-Analysis of Research (DeLMAR), School of Business and Law, Deakin University, Burwood, Victoria, Australia.
Research Synthesis Methods
|October 10, 2021
Summary
Many research studies suffer from low statistical power, leading to false positives. High median retrospective power (MRP) indicates credible findings, while low MRP suggests caution is needed for meta-analysis results.
Area of Science:
- Psychology
- Reproducibility in Science
- Research Methodology
Background:
- Recent large-scale, preregistered replication failures highlight issues with 'false positives' in scientific research.
- Inadequate statistical power is a widespread problem, compromising the reliability of experimental results.
- Publication bias and selective reporting further inflate the rate of false positives in the research record.
Purpose of the Study:
- To investigate the relationship between statistical power and the rate of false positives in scientific literature.
- To determine the threshold of median retrospective power (MRP) associated with credible research findings and successful replications.
Main Methods:
- Analysis of large-scale, preregistered replication studies.
- Examination of meta-analyses in relation to statistical power.
- Calculation of median retrospective power (MRP) across studies.
Main Results:
- High median retrospective power (MRP >50%-60%) is linked to credible meta-analyses and successful large-scale replications.
- Low median retrospective power (MRP <50%) is associated with a higher likelihood of false positive findings.
- The study establishes a critical threshold for MRP in evaluating research credibility.
Conclusions:
- Median retrospective power (MRP) is a crucial indicator of research credibility.
- Findings from meta-analyses with low MRP should be viewed with significant skepticism.
- Ensuring adequate statistical power is essential for the integrity of scientific research and replication efforts.
Related Concept Videos
01:52Regression Toward the Mean
6.6K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.6K
01:13Accuracy and Errors in Hypothesis Testing
374
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
374
01:29Bias in Epidemiological Studies
806
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
806
01:22Bias
6.3K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
6.3K
01:11Statistical Analysis: Overview
9.9K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
9.9K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
201
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
201