Related Experiment Video
Updated: Aug 12, 2025

Author Spotlight: Biological Standardization to Ensure Reproducibility and Harmonization in Research
Published on: August 4, 2023
What's in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of
Sophia Crüwell1,2, Deborah Apthorp3,4, Bradley J Baker5
1Meta-Research Innovation Center Berlin (METRIC-B), QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité - Universitätsmedizin Berlin.
Abstract:
In April 2019, Psychological Science published its first issue in which all Research Articles received the Open Data badge. We used that issue to investigate the effectiveness of this badge, focusing on the adherence to its aim at Psychological Science: sharing both data and code to ensure reproducibility of results. Twelve researchers of varying experience levels attempted to reproduce the results of the empirical articles in the target issue (at least three researchers per article). We found that all 14 articles provided at least some data and six provided analysis code, but only one article was rated to be exactly reproducible, and three were rated as essentially reproducible with minor deviations. We suggest that researchers should be encouraged to adhere to the higher standard in force at Psychological Science. Moreover, a check of reproducibility during peer review may be preferable to the disclosure method of awarding badges.
Related Concept Videos
Ethics in Research
Social Proof
Psychology as a Science
The scientific method in psychology involves six critical steps: making observations, formulating hypotheses, conducting tests, analyzing...
Reliability and Validity
Stereotypes, Prejudice, and Discrimination
Self-Discrepancy Theory

