Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

A comment on replication, p-values and evidence.

S N Goodman1

  • 1Johns Hopkins University School of Medicine, Department of Oncology, Baltimore, MD 21205.

Statistics in Medicine
|May 1, 1992
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Colorectal cancer in ulcerative colitis: survival in patients with and without colorectal cancer symptoms.

Inflammatory bowel diseases·2013
Same author

WITHDRAWN: Anticoagulants or antiplatelet therapy for non-rheumatic atrial fibrillation and flutter.

The Cochrane database of systematic reviews·2007
Same author

Left insular stroke is associated with adverse cardiac outcome.

Neurology·2006
Same author

Pancreatic cancer after remote peptic ulcer surgery.

Journal of clinical pathology·2002
Same author

Management of acne.

Evidence report/technology assessment (Summary)·2002
Same author

Anticoagulants or antiplatelet therapy for non-rheumatic atrial fibrillation and flutter.

The Cochrane database of systematic reviews·2001
Same journal

Interpretable Bayesian Modeling for Multireader Multicase Studies: Addressing Overdispersion and Limited Sample Size in Diagnostic Enhancement Evaluation.

Statistics in medicine·2026
Same journal

Adaptive Sequential Multiple Hypotheses Testing for Concomitant Vaccine Safety Surveillance.

Statistics in medicine·2026
Same journal

Novel Distance Regression for Repeated Outcomes With Missing Data: Applications to Longitudinal and Crossover Studies of Microbiome Beta-Diversity.

Statistics in medicine·2026
Same journal

Optimal Weighted Tests for Replication Studies and the 'Two-Trials Rule' With Multiple Hypotheses.

Statistics in medicine·2026
Same journal

Identifiable Copula-Double-Cox Models: A Fully Parametric Framework for Dependent Right-Censored Survival Data.

Statistics in medicine·2026
Same journal

Moving From Individualized Risk-Based Prevention to Benefit-Based Prevention: Estimating Individualized Life-Years Gained From Prevention Services as a Basis for Eligibility.

Statistics in medicine·2026
See all related articles

Small p-values do not guarantee reproducible results. This study reveals that the probability of replicating a statistically significant finding is often lower than expected, challenging conventional interpretations of statistical significance.

Area of Science:

  • Statistics
  • Scientific Methodology

Background:

  • Conventional interpretation suggests small p-values indicate high credibility for alternative hypotheses.
  • It is commonly believed that significant results are likely to be replicated in subsequent experiments.

Purpose of the Study:

  • To investigate the actual probability of replicating statistically significant findings.
  • To identify the source of overestimation in the credibility of alternative hypotheses based on p-values.
  • To propose the 'replication probability' as a frequentist measure for evaluating evidence.

Main Methods:

  • Analysis of the interpretation of post-trial p-values compared to pre-trial alpha error.
  • Development and application of the 'replication probability' concept.

Related Experiment Videos

Main Results:

  • The probability of replicating a statistically significant result, when the observed difference is true, is substantially lower than conventionally expected.
  • Misinterpretation of post-trial p-values as pre-trial error rates contributes to this discrepancy.
  • P-values tend to overstate the evidence against the null hypothesis.

Conclusions:

  • The common understanding of p-values and statistical significance requires revision.
  • The 'replication probability' offers a more realistic frequentist measure of evidence.
  • Researchers should be cautious about over-interpreting the certainty conveyed by small p-values.