Reproducibility in the Social Sciences.

Area of Science:

• Social Sciences
• Research Methodology
• Academic Publishing

Background:

• Growing concern regarding the reproducibility of empirical research in social sciences.
• Vast and expanding literature on research reproducibility makes it challenging for new researchers to grasp the field.
• Need for a comprehensive overview of the reproducibility literature to identify core themes and disciplinary differences.

Purpose of the Study:

• To formally characterize the field of social science reproducibility using text modeling.
• To summarize the breadth of the reproducibility literature and identify its core themes.
• To reveal disciplinary differences and the diversity of subtopics within reproducibility research.

Main Methods:

• Construction and analysis of text networks from 1,947 articles on research reproducibility.
• Application of formal text modeling to synthesize a large body of academic literature.
• Comparative analysis of reproducibility publications across different social science disciplines.

Main Results:

• Identified heterogeneity in the reproducibility problem, with multiple error sources and solution strategies.
• Revealed significant differences in reproducibility concerns and approaches across social science disciplines.
• Found that the literature addresses a wide diversity of subtopics within the broader reproducibility challenge.

Conclusions:

• Reproducibility in social sciences is not a monolithic issue; solutions require nuanced, discipline-specific approaches.
• Calls for passive remedies like open science may be insufficient due to the heterogeneous nature of reproducibility problems.
• Proposed an alternative model emphasizing active, pre-publication rigor to enhance research reliability more effectively.