The natural selection of bad science

Area of Science:

• Behavioral Sciences
• Scientific Methodology
• Research Culture

Background:

• Poor research design and data analysis contribute to a high rate of false-positive findings in scientific publications.
• Despite repeated calls for improvement, statistical power in behavioral sciences has not increased over 60 years, indicating a systemic issue beyond mere misunderstanding.

Purpose of the Study:

• To investigate the role of institutional incentives in the persistence of poor scientific methods.
• To demonstrate how career advancement structures can inadvertently select for suboptimal research practices and increase false discovery rates.

Main Methods:

• A 60-year meta-analysis of statistical power in behavioral sciences was conducted.
• A computational model simulating scientific communities was developed to explore the dynamics of method selection and career advancement.

Main Results:

• The meta-analysis revealed no improvement in statistical power over six decades.
• The computational model demonstrated that selection for high publication output leads to methodological deterioration and increased false discovery rates.
• Replication was found to slow, but not halt, this decline in research quality.

Conclusions:

• The persistence of poor scientific methods is driven by incentives that prioritize publication quantity over research quality and genuine understanding.
• Systemic change at the institutional level, shifting rewards towards understanding rather than mere publication, is necessary to improve the culture of science.