Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Confidence intervals and replication: where will the next mean fall?

Geoff Cumming1, Robert Maillardet

  • 1School of Psychological Science, La Trobe University, Melbourne, Australia. g.cumming@latrobe.edu.au

Psychological Methods
|September 7, 2006
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

From significance testing to estimation and Open Science: How esci can help.

International journal of psychology : Journal international de psychologie·2024
Same author

The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices: Five years later.

Behavior research methods·2022
Same author

The diamond ratio: A visual indicator of the extent of heterogeneity in meta-analysis.

The British journal of mathematical and statistical psychology·2021
Same author

The frequent insignificance of a "significant" p-value.

Journal of cardiac surgery·2021
Same author

Replication: Do not trust your p-value, be it small or large.

The Journal of physiology·2021
Same author

A Tribute to the Mind, Methodology and Mentoring of Wayne Velicer.

Multivariate behavioral research·2020
Same journal

Bayesian evaluation for latent variable models: A tutorial on computing information criteria and bayes factors with the r package bleval.

Psychological methods·2026
Same journal

A stochastic block prior for clustering in graphical models.

Psychological methods·2026
Same journal

Three-level vector autoregressive models.

Psychological methods·2026
Same journal

Scaling cognitive modeling to big data: A deep learning approach to studying individual differences in evidence accumulation model parameters.

Psychological methods·2026
Same journal

Best practices in multilevel modeling for within-cluster group comparisons: An evaluation of coding strategies reflecting group composition and heterogeneity.

Psychological methods·2026
Same journal

A unified framework for psychometrics in experimental psychology: The standardized generalized hierarchical factor model.

Psychological methods·2026
See all related articles

Confidence intervals (CIs) offer insights into study replication, but their interpretation is often misunderstood. Most 95% CIs capture substantial replication means, yet variability exists, impacting the reliability of future research findings.

Area of Science:

  • Statistics
  • Research Methodology
  • Psychometrics

Background:

  • Confidence intervals (CIs) are crucial for interpreting statistical significance and study replication.
  • Many researchers hold misconceptions regarding the information CIs provide about future replication means.
  • The percentage of future replication means captured by a CI varies depending on its position relative to the population mean.

Purpose of the Study:

  • To investigate the distribution of the percentage of future replication means captured by confidence intervals.
  • To examine this distribution across different conditions, including known/unknown population variance, sample sizes, and robust CIs.
  • To enhance understanding of what confidence intervals convey about replication and extend the discussion to p-values.

Main Methods:

Related Experiment Videos

  • The study analyzed the distribution of the percentage of future replication means captured by confidence intervals.
  • Investigated scenarios with known and unknown population variance (sigma).
  • Examined various sample sizes and employed robust confidence intervals.

Main Results:

  • The distribution of captured replication means exhibits a strong negative skew.
  • Most 95% confidence intervals capture approximately 90% or more of future replication means.
  • On average, a 95% confidence interval includes only 83.4% of future replication means, with considerable variation.

Conclusions:

  • Confidence intervals provide valuable but imperfect information about replication.
  • Researchers must be aware of the skewed distribution and average capture rates to avoid misinterpretation.
  • Visual aids and a nuanced understanding of CIs and p-values are essential for accurate research interpretation.