A large-scale in silico replication of ecological and evolutionary studies

Area of Science:

• Ecological sciences
• Evolutionary biology
• Scientific methodology

Background:

• Growing concerns regarding the replicability of ecological and evolutionary studies.
• Lack of field-wide replication projects to assess current standards.

Purpose of the Study:

• To conduct a large-scale in silico replication project in ecology and evolution.
• To evaluate the impact of statistical significance on study replicability.
• To identify factors influencing the credibility and transparency of research.

Main Methods:

• Utilized cutting-edge statistical methodologies for an in silico replication project.
• Analyzed the replicability of studies based on the strength of evidence against the null hypothesis (H0).
• Compared sample size requirements for achieving different levels of replicability.

Main Results:

• Replicability rates were 30%-40% for studies with marginal statistical significance.
• Studies presenting strong evidence against H0 demonstrated >70% replicability.
• Marginal significance studies require sevenfold larger sample sizes to match the replicability of strong evidence studies.

Conclusions:

• A significant portion of ecological and evolutionary studies may lack sufficient statistical power for reliable replication.
• The strength of statistical evidence is a critical determinant of replicability.
• Advocates for a paradigm shift towards transparent, credible, and replicable research planning, execution, and publication in ecology and evolution.