Evaluating replicability of laboratory experiments in economics

Area of Science:

• Economics
• Econometrics
• Scientific Replication

Background:

• Recent concerns regarding the replicability of scientific findings.
• The need for empirical data on replicability within the field of economics.

Purpose of the Study:

• To assess the replicability of empirical studies published in top economics journals.
• To provide quantitative evidence on the reproducibility of economic research.

Main Methods:

• Replication of 18 studies from the American Economic Review and Quarterly Journal of Economics (2011-2014).
• Utilization of predefined, publicly available analysis plans for each replication.
• Ensuring all replications had at least 90% statistical power to detect original effect sizes at a 5% significance level.

Main Results:

• Significant effects in the original direction were found in 11 out of 18 replications (61%).
• The average effect size in successful replications was 66% of the original reported effect size.
• Additional replicability indicators, including prediction markets, showed rates between 67% and 78%.

Conclusions:

• A substantial portion of economic studies exhibit moderate replicability.
• The findings contribute empirical evidence to the ongoing discourse on scientific reproducibility in economics.
• The study highlights the importance of transparent analysis plans and adequate statistical power in ensuring reliable research findings.