Perceived crisis and reforms: Issues, explanations, and remedies

Area of Science:

• Psychology
• Research Methodology

Background:

• The scientific community faces a perceived replication crisis.
• This crisis impacts the reliability and reproducibility of research findings.

Purpose of the Study:

• To provide an overview of the replication crisis, examining its problems, explanations, and potential remedies.
• To analyze the role of small and varying effect sizes in the crisis.

Main Methods:

• Review of large-scale replication studies.
• Analysis of meta-analysis literature.
• Discussion of research practices and institutional factors.

Main Results:

• Many scientific effects are small, with variance often 10-25% of population variance, likely due to context dependencies.
• Questionable research practices (e.g., low power, poor measurement, null hypothesis significance testing) and systemic factors contribute.
• Small, varying effect sizes inherent to psychological reality present significant challenges.

Conclusions:

• Remedies include increasing power, conducting more replications and meta-analyses, improving reliability, and exploring alternatives to null hypothesis significance testing (NHST).
• A diversity of perspectives and methods is needed.
• Complexity and context-dependency of human behavior necessitate adjustments in research and analysis methods.