Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

Area of Science:

• Ecology and Evolutionary Biology
• Computational Reproducibility
• Meta-Research

Background:

• Journals increasingly encourage data and code sharing in ecology and evolutionary biology.
• Ensuring computational reproducibility is crucial for scientific integrity and advancement.
• Previous assessments of data and code sharing practices are limited.

Purpose of the Study:

• To assess the extent of data and code sharing in ecology and evolutionary biology meta-analyses.
• To determine the success rate of computationally reproducing published results using shared data and code.
• To establish a benchmark for computational reproducibility in the field.

Main Methods:

• Surveyed 177 meta-analyses published in ecology and evolutionary biology journals (2015-2017).
• Categorized articles based on data and code sharing practices (data only, code only, both).
• Attempted computational reproduction of targeted results from 26 articles that shared both data and code.

Main Results:

• 60% of articles shared data only, 1% shared code only, and 15% shared both.
• Successful computational reproduction of targeted results ranged from 27% to 73% across the 26 articles.
• Variability in reproduction success depended on the stringency of criteria used.

Conclusions:

• A significant gap exists between data/code sharing and actual computational reproducibility.
• Current data and code sharing practices do not consistently enable independent verification of published results.
• Further improvements in data and code sharing are needed to enhance the reliability of ecological research.