Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Accuracy and Precision01:52

Accuracy and Precision

Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.  Highly accurate measurements...
Accuracy and Precision01:52

Accuracy and Precision

Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.  Highly accurate measurements...
Uncertainty in Measurement: Accuracy and Precision03:37

Uncertainty in Measurement: Accuracy and Precision

Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
P-value01:10

P-value

P-value is one of the most crucial concepts in statistics.
P-value stands for the probability value.  P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to  not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more unlikely...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Pairwise Comparisons of <math><semantics><mrow><mi>k</mi></mrow> <annotation>$$ k $$</annotation></semantics></math> Binomial Responses.

Statistics in medicine·2026
Same author

Assessing Tuning Parameter Selection Variability in Penalized Regression.

Technometrics : a journal of statistics for the physical, chemical, and engineering sciences·2019
Same author

Variable Selection in Kernel Regression Using Measurement Error Selection Likelihoods.

Journal of the American Statistical Association·2018
Same author

Interactive Q-learning for Quantiles.

Journal of the American Statistical Association·2017
Same author

iqLearn: Interactive Q-Learning in R.

Journal of statistical software·2016
Same author

Assessing Variability of Complex Descriptive Statistics in Monte Carlo Studies using Resampling Methods.

International statistical review = Revue internationale de statistique·2015
Same journal

Conceptualizing Experimental Controls Using the Potential Outcomes Framework.

The American statistician·2025
Same journal

A Cornucopia of Maximum Likelihood Algorithms.

The American statistician·2025
Same journal

A Multiple Imputation Approach for the Cumulative Incidence, with Implications for Variance Estimation.

The American statistician·2025
Same journal

An Example to Illustrate Randomized Trial Estimands and Estimators.

The American statistician·2025
Same journal

Laplace's law of succession estimator and M-statistics.

The American statistician·2025
Same journal

Counternull sets in randomized experiments.

The American statistician·2025
See all related articles

Related Experiment Video

Updated: May 21, 2026

Mapping Dysfunctional Protein-Protein Interactions in Disease
09:39

Mapping Dysfunctional Protein-Protein Interactions in Disease

Published on: October 24, 2025

P-Value Precision and Reproducibility.

Dennis D Boos1, Leonard A Stefanski

  • 1Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203.

The American Statistician
|June 13, 2012
PubMed
Summary
This summary is machine-generated.

Statistical p-values show significant sample-to-sample variability, impacting result reproducibility. Understanding this variability is crucial for interpreting research findings and p-value precision.

More Related Videos

A Validatable Droplet Digital Polymerase Chain Reaction Assay for the Detection of Adeno-Associated Viral Vectors in Bioshedding Studies of Tears
07:43

A Validatable Droplet Digital Polymerase Chain Reaction Assay for the Detection of Adeno-Associated Viral Vectors in Bioshedding Studies of Tears

Published on: July 14, 2023

Related Experiment Videos

Last Updated: May 21, 2026

Mapping Dysfunctional Protein-Protein Interactions in Disease
09:39

Mapping Dysfunctional Protein-Protein Interactions in Disease

Published on: October 24, 2025

A Validatable Droplet Digital Polymerase Chain Reaction Assay for the Detection of Adeno-Associated Viral Vectors in Bioshedding Studies of Tears
07:43

A Validatable Droplet Digital Polymerase Chain Reaction Assay for the Detection of Adeno-Associated Viral Vectors in Bioshedding Studies of Tears

Published on: July 14, 2023

Area of Science:

  • Statistics
  • Statistical Inference
  • Reproducibility in Science

Background:

  • P-values are standard measures of evidence against a null hypothesis in statistical testing.
  • The sample-to-sample variability of p-values is often overlooked, limiting a full appreciation of their uncertainty.
  • This oversight can contribute to issues with the reproducibility of scientific results.

Purpose of the Study:

  • To systematically investigate and quantify the variability of p-values across different data scenarios.
  • To provide context for the challenges in replicating statistical findings.
  • To evaluate the precision of commonly reported p-value significance levels (*, **, ***).

Main Methods:

  • Analysis of log-scale p-value standard errors.
  • Application of bootstrap prediction bounds for estimating future p-value ranges.
  • Calculation of reproducibility probabilities for future replicate p-values.

Main Results:

  • P-values demonstrate surprisingly large variability in typical data situations.
  • This inherent variability offers an explanation for the frequent failure of statistical results to replicate.
  • The precision of reporting p-values using significance stars (*, **, ***) aligns with standard rounding rules.

Conclusions:

  • The substantial variability of p-values necessitates careful consideration in statistical interpretation.
  • Understanding p-value variability is key to addressing the reproducibility crisis in science.
  • Current practices for reporting p-value significance levels appear appropriate given their variability.