Care to share? Experimental evidence on code sharing behavior in the social sciences.

Area of Science:

• Social Sciences
• Research Methodology
• Scientific Transparency

Background:

• Replication and reproduction are essential for scientific integrity.
• Sharing data and analysis code is crucial for enabling replications, especially in complex large-N observational studies.
• Understanding factors influencing researchers' willingness to share code is vital for promoting open science.

Purpose of the Study:

• To investigate factors influencing researchers' code sharing behavior upon request.
• To experimentally test the impact of request framing, appeal type, and perceived effort on code sharing.
• To identify effective strategies for increasing the availability of research code.

Main Methods:

• A preregistered multifactorial field experiment involving 1,206 authors who published using European Social Survey data (2015-2020).
• Email requests with randomly varied framing (enhancement vs. replication crisis), appeal (FAIR, altruism, citation, none), and perceived effort (cleaning required/not required).
• Analysis of code sharing rates based on experimental treatments.

Main Results:

• Overall, 37.5% of contacted authors shared their analysis code.
• Request framing significantly influenced code sharing; mentioning the replication crisis increased sharing rates.
• Appeals to FAIR principles, academic altruism, or citation prospects, and information on code cleaning effort, did not significantly affect sharing behavior.

Conclusions:

• Individual, small-scale interventions are insufficient to significantly enhance research code availability.
• Framing requests around the replication crisis can positively impact code sharing, but its effect is limited.
• Systemic, large-scale institutional norms and policies are necessary to foster widespread research code sharing and improve scientific transparency.