Jove
Visualize
Contact Us

Related Experiment Videos

Replication is about effect size: Comment on Maxwell, Lau, and Howard (2015).

Warren W Tryon1

  • 1Fordham University.

The American Psychologist
|April 5, 2016
PubMed
Summary

Null hypothesis testing offers a false promise of single-study reliability. Researchers should focus on replicating effect sizes and narrowing confidence intervals to assess psychological effect replicability using cumulative meta-analyses.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Explanation and Psychological Mechanisms.

Psychological reports·2026
Same author

The Personal Identity Test: A Measure of Discontinuity in Core Personal Identity.

Psychological reports·2025
Same author

Unconscious processing: Comment on Hornsey and Fielding (2017).

The American psychologist·2018
Same author

Basing clinical practice on unified psychological science: Comment on Melchert (2016).

The American psychologist·2017
Same author

Integrating psychology and neuroscience: Comment on Schwartz et al. (2016).

The American psychologist·2016
Same author

Underreliance on mechanistic models: Comment on Ferguson (2015).

The American psychologist·2016
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Area of Science:

  • Psychology
  • Statistical Methods

Background:

  • Null hypothesis testing (NHT) is widely used but can provide a misleading sense of certainty.
  • The reliability of single studies is often overestimated, leading to a false promise of trust.

Purpose of the Study:

  • To challenge the reliance on single studies for establishing scientific truth.
  • To propose a more robust approach for evaluating the replicability of research findings.
  • To highlight the utility of cumulative meta-analyses in assessing effect size reproducibility.

Main Methods:

  • Critiquing the traditional approach of null hypothesis testing.
  • Advocating for a focus on effect size reproducibility and confidence interval precision.
  • Emphasizing the application of cumulative meta-analyses for ongoing research synthesis.

Related Experiment Videos

Main Results:

  • Single studies, particularly those relying solely on NHT, do not guarantee reliable findings.
  • Replicability is better assessed by the consistency of effect sizes across studies.
  • Narrowing confidence intervals indicates increasing precision and reliability of effect size estimates.

Conclusions:

  • The trustworthiness of scientific findings hinges on replicability, not isolated statistical significance.
  • Cumulative meta-analyses are essential tools for understanding the robustness and magnitude of psychological effects.
  • Future research should prioritize methods that enhance the reproducibility of effect sizes.