Related Experiment Video
Updated: Jul 4, 2026
06:26
Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI)
Published on: November 27, 2019
Accuracy of effect size estimates from published psychological research
Andrew Brand1, Michael T Bradley, Lisa A Best
1King's College London, Institute of Psychiatry.
Perceptual and Motor Skills
|June 18, 2008
Summary
Publication bias and low statistical power inflate effect sizes in research. This simulation shows how this overestimation impacts meta-analyses and power analyses, suggesting mitigation strategies.
Area of Science:
- Biostatistics
- Research Methodology
- Scientific Publishing
Background:
- Publication bias, favoring statistically significant results, is prevalent in scientific literature.
- Inadequate statistical power in studies can exacerbate bias.
- Overestimated effect sizes have significant implications for research synthesis and future study design.
Purpose of the Study:
- To model the biasing of effect sizes in published studies using Monte Carlo simulation.
- To investigate the consequences of overestimated effect sizes on meta-analyses and power analyses.
- To identify potential measures for reducing the problem of effect size overestimation.
Main Methods:
- Employed Monte Carlo simulation to model the phenomenon of effect size bias.
- Focused on scenarios combining publication bias with insufficient statistical power.
- Analyzed the downstream effects on statistical analyses commonly used in research synthesis.
Main Results:
- The simulation demonstrated that a combination of publication bias and low statistical power leads to an overestimation of effect sizes.
- This overestimation was shown to have a direct negative impact on the accuracy of meta-analyses.
- The reliability of power analyses is also compromised by inflated effect size estimates.
Conclusions:
- Overestimation of effect sizes is a significant issue in published research, driven by publication bias and low power.
- This bias distorts findings in meta-analyses and compromises the validity of power calculations.
- Implementing strategies to mitigate publication bias and improve statistical power is crucial for research integrity.
Related Concept Videos
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:37Statistical Significance
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
01:19Testing a Claim about Standard Deviation
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
02:45Self-Discrepancy Theory
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
01:06What are Estimates?
It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates.
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such as the mean,...
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such as the mean,...