A simple model suggesting economically rational sample-size choice drives irreproducibility

Area of Science:

• Scientific reproducibility
• Research methodology
• Economics of science

Background:

• Systematic studies indicate a significant portion of published research lacks reproducibility.
• Insufficient sample size is a probable cause, leading to low statistical power and positive predictive value.
• Scientific competition and positive publication bias are hypothesized drivers of inadequate sample-size selection.

Purpose of the Study:

• To formalize the intuition that economic rationality influences sample-size choices in research.
• To develop a model predicting sample sizes based on the balance between experimentation costs and publication income.
• To explain the prevalence and persistence of small sample sizes and their link to irreproducibility.

Main Methods:

• Formalization of economic rationality in sample-size selection using a simple model.
• Integration of basic statistical formulas with economic optimality predictions.
• Analysis of how factors like effect probability, effect size, and grant income impact rational sample size.

Main Results:

• Economically rational sample sizes are small when effects have low base probability, small effect size, or low grant income per publication.
• The model robustly predicts a bimodal distribution of statistical power and low reproducibility rates, aligning with empirical findings.
• Conditional equivalence testing is proposed as a strategy to align economic incentives with adequate sample sizes.

Conclusions:

• Economic rationality, or pressures, is a principal driver of research irreproducibility.
• The model provides a simple mechanism explaining small sample sizes and suggests avenues for empirical validation.
• Strategies to realign economic incentives are crucial for improving scientific reproducibility.