Assessing data availability and research reproducibility in hydrology and water resources

Area of Science:

• Hydrology and Water Resources
• Scientific Publishing
• Research Reproducibility

Background:

• Improving the reproducibility of published research is a significant concern across scientific disciplines.
• The availability of digital research artifacts (data, models, code) is essential for verifying study outcomes.

Purpose of the Study:

• To develop and apply a survey tool to assess the availability of digital research artifacts and the reproducibility of results in hydrology and water resources articles.
• To identify key bottlenecks hindering research reproducibility in the field.

Main Methods:

• A survey tool was developed to evaluate digital artifact availability and result reproducibility.
• The tool was used to assess 360 articles from six hydrology and water resources journals published in 2017.
• Reproducibility was tested using the available digital artifacts.

Main Results:

• Only 1.6% of the tested articles had their results reproduced using available artifacts.
• An estimated 0.6% to 6.8% of all articles might have reproducible results.
• Key bottlenecks identified include incomplete artifact availability (44%), lack of directions (89%), and non-reproducible results despite artifact availability (5%).

Conclusions:

• The developed survey tool effectively identifies critical barriers to research reproducibility in hydrology.
• The findings underscore the need for improved practices in sharing digital research artifacts and ensuring result reproducibility.
• The tool can assist authors, journals, and institutions in assessing and enhancing research reproducibility.