Related Experiment Video
Updated: Jun 22, 2026
08:29
An Open Source Technology Platform to Manufacture Hydrogel-Based 3D Culture Models in an Automated and Standardized Fashion
Published on: March 31, 2022
4.7K
Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: an
Tom E Hardwicke1,2, Manuel Bohn3, Kyle MacDonald4
1Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.
Royal Society Open Science
|February 22, 2021
Summary
Ensuring scientific reproducibility is crucial. A study found that even with open data, many psychological science articles had numerical discrepancies, highlighting issues with unclear reporting rather than data accessibility alone.
Area of Science:
- Psychology
- Scientific Methodology
Background:
- Analytic reproducibility is a cornerstone of scientific integrity.
- Open data initiatives aim to enhance research transparency and verifiability.
Purpose of the Study:
- To assess the extent of analytic reproducibility in psychological science articles.
- To identify factors contributing to or hindering reproducibility.
Main Methods:
- Reviewed 25 Psychological Science articles published between 2014-2015 with open data badges.
- Attempted to reproduce original analyses and outcomes.
- Contacted original authors for clarification when discrepancies arose.
Main Results:
- Initially, 64% of articles had major numerical discrepancies (>10% difference).
- Only 36% were reproducible without author input; an additional 24% required author involvement.
- Major numerical discrepancies were found in 5% of checked values.
- Unclear reporting of analytic procedures was the primary cause of non-reproducibility.
Conclusions:
- Open data alone is insufficient to guarantee analytic reproducibility.
- Clear and detailed reporting of analytical methods is essential for scientific verification.
- Further efforts are needed to improve the transparency and reproducibility of psychological research.
Related Concept Videos
02:30Naturalistic Observations
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
01:56Ethics in Research
Today, scientists agree that good research is ethical in nature and is guided by a basic respect for human dignity and safety. However, this has not always been the case. Modern researchers must demonstrate that the research they perform is ethically sound.
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:11Observational Studies
Observational studies are a type of analytical study where researchers observe events without any interventions. In other words, the researcher does not influence the response variable or the experiment's outcome.
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
01:13Psychology as a Science
Psychology, as a scientific discipline, aims to understand the mind and behavior through rigorous and systematic methods. The foundation of psychological research is evidence-based, relying heavily on the scientific method to derive and validate knowledge. This structured approach ensures that findings are reliable, valid, and applicable to broader contexts.
The scientific method in psychology involves six critical steps: making observations, formulating hypotheses, conducting tests, analyzing...
The scientific method in psychology involves six critical steps: making observations, formulating hypotheses, conducting tests, analyzing...
01:30Scientific Nature of Social Psychology
Social psychology is a scientific discipline dedicated to understanding how individuals think, feel, and behave in social contexts. Unlike common sense, which relies on anecdotal experiences and intuition, social psychology employs systematic research and empirical methods to ensure objectivity and reliability. This distinction is fundamental in distinguishing scientifically supported findings from mere speculation.Four fundamental scientific values guide a structured approach to research in...