Investigating the replicability of the social and behavioural sciences

Area of Science:

• Social and behavioral sciences
• Quantitative psychology
• Replication science

Background:

• Replicability is crucial for establishing generalizable knowledge in science.
• Previous research has highlighted concerns regarding the reproducibility of scientific findings.
• The social and behavioral sciences are a major area for research with broad societal implications.

Purpose of the Study:

• To assess the replicability of positive results from quantitative studies in social and behavioral sciences.
• To quantify the extent of replication success across a large set of claims and papers.
• To investigate potential variations in replication rates across different disciplines within these fields.

Main Methods:

• Replication of 274 claims from 164 quantitative papers published between 2009-2018 in 54 journals.
• High-powered replications (median power of 99.6%) using original materials where available.
• Peer review of replication studies in advance via a standardized internal protocol.

Main Results:

• Replication success was observed for 55.1% of claims and 49.3% of papers.
• A significant reduction in effect size was found, with median Pearson's r decreasing from 0.25 to 0.10.
• Replication rates varied modestly across disciplines (42.5%-63.1%), with some estimates having high uncertainty.

Conclusions:

• Challenges to replicability are prevalent across the social and behavioral sciences.
• The findings underscore the importance of identifying factors that influence scientific reproducibility.
• Further research is needed to understand and enhance the conditions that promote or inhibit replicability.