Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge.

Area of Science:

• Computational social science
• Research methodology
• Data science

Background:

• Reproducibility is crucial for scientific integrity.
• Computational reproducibility, recreating results with original data and code, remains a practical challenge for researchers.
• The Fragile Families Challenge highlights the need for robust reproducibility methods.

Purpose of the Study:

• To present a practical approach for enhancing computational reproducibility in academic research.
• To demonstrate the application of software engineering tools for reproducible scientific studies.
• To facilitate the replication of findings from the 12 articles in the Socius special issue on the Fragile Families Challenge.

Main Methods:

• Utilized software containers (e.g., Docker) to standardize computational environments.
• Leveraged cloud computing platforms (e.g., Amazon Web Services) for scalable and consistent research execution.
• Applied these tools across 12 research articles for the Fragile Families Challenge special issue.

Main Results:

• Successfully standardized the computing environment for each submitted article.
• Enabled easier computational reproducibility for the Fragile Families Challenge studies.
• Demonstrated the feasibility of using professional software engineering tools in academic research.

Conclusions:

• Software containers and cloud computing significantly improve computational reproducibility.
• Recommendations are provided for researchers and journals to adopt these tools for future studies.
• Standardizing environments ensures research integrity and facilitates future scientific advancements.