Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Randomized Experiments01:13

Randomized Experiments

7.2K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
7.2K
Controls in Experiments01:13

Controls in Experiments

11.2K
When conducting an experiment, it is crucial to have control to reduce bias and accurately measure the dependent variables. It also marks the results more reliable. Controls are elements in an experiment that have the same characteristics as the treatment groups but are not affected by the independent variable. By sorting these data into control and experimental conditions, the relationship between the dependent and independent variables can be drawn. A randomized experiment always includes a...
11.2K
Null and Alternative Hypotheses01:16

Null and Alternative Hypotheses

10.0K
The actual hypothesis testing begins by considering two hypotheses. They are termed  the null hypothesis and the alternative hypothesis. These hypotheses contain opposing viewpoints.
The null hypothesis, denoted by H0 is a statement of no difference between the variables—they are not related. This can often be considered the status quo. As  a result if you cannot accept the null, it requires some action.
The alternative hypothesis, denoted by H1 or Ha, is a claim about the...
10.0K
Wald-Wolfowitz Runs Test II01:17

Wald-Wolfowitz Runs Test II

317
The Wald-Wolfowitz runs test, commonly referred to as the runs test, is a nonparametric test used to assess the randomness of ordered data. The test evaluates the number of runs, which are consecutive sequences of similar elements within the data. If the number of runs is significantly higher or lower than expected, the data is considered non-random, indicating a detectable pattern or structure.
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and...
317
Wald-Wolfowitz Runs Test I01:17

Wald-Wolfowitz Runs Test I

742
The Wald-Wolfowitz test, also known as the runs test, is a nonparametric statistical test used to assess the randomness of a sequence of two different types of elements (e.g., positive/negative values, successes/failures). It examines whether the order of the elements in a sequence is random or if there is a pattern or trend present. This nonparametric test applies to any ordered data despite the population and sample data distribution, even if a higher sample size is available.
The test works...
742
Random Variables01:09

Random Variables

13.4K
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
13.4K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

High-dimensional randomization-based inference capitalizing on classical design and modern computing.

Behaviormetrika·2023
Same author

The importance of having a conceptual stage when reporting non-randomized studies.

Biostatistics & epidemiology·2021
Same author

Heterogeneous ozone effects on the DNA methylome of bronchial cells observed in a crossover study.

Scientific reports·2020
Same author

When possible, report a Fisher-exact <i>P</i> value and display its underlying null randomization distribution.

Proceedings of the National Academy of Sciences of the United States of America·2020
Same author

Causal mediation analysis for longitudinal data with exogenous exposure.

Biostatistics (Oxford, England)·2015
Same author

Estimation of causal effects of binary treatments in unconfounded studies with one continuous covariate.

Statistical methods in medical research·2015
Same journal

Conceptualizing Experimental Controls Using the Potential Outcomes Framework.

The American statistician·2025
Same journal

A Cornucopia of Maximum Likelihood Algorithms.

The American statistician·2025
Same journal

A Multiple Imputation Approach for the Cumulative Incidence, with Implications for Variance Estimation.

The American statistician·2025
Same journal

An Example to Illustrate Randomized Trial Estimands and Estimators.

The American statistician·2025
Same journal

Laplace's law of succession estimator and M-statistics.

The American statistician·2025
Same journal

Flexible Distributed Lag Models for Count Data Using mgcv.

The American statistician·2025
See all related articles

Related Experiment Video

Updated: Sep 12, 2025

A Within-Subject Experimental Design using an Object Location Task in Rats
09:28

A Within-Subject Experimental Design using an Object Location Task in Rats

Published on: May 6, 2021

4.6K

Counternull sets in randomized experiments.

M-A C Bind1,2, D B Rubin3,4

  • 1Biostatistics Center, Massachusetts General Hospital, Boston, MA, USA.

The American Statistician
|August 8, 2025
PubMed
Summary
This summary is machine-generated.

Researchers often incorrectly conclude "no effect" from non-significant results. Reporting "counternull" values, which have equal statistical evidence as the null hypothesis, can prevent this misinterpretation in studies.

Keywords:
Fisher-exact p-valueHypothesis testingRandomization inferenceRandomization-based inference

More Related Videos

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K
The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.5K

Related Experiment Videos

Last Updated: Sep 12, 2025

A Within-Subject Experimental Design using an Object Location Task in Rats
09:28

A Within-Subject Experimental Design using an Object Location Task in Rats

Published on: May 6, 2021

4.6K
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K
The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.5K

Area of Science:

  • Statistics
  • Biostatistics
  • Clinical Trial Methodology

Background:

  • Studies with non-statistically significant primary results are frequently misinterpreted as evidence of no effect.
  • This misinterpretation can lead to flawed conclusions in scientific literature and clinical practice.

Purpose of the Study:

  • To introduce and advocate for the reporting of "counternull" values alongside traditional null hypothesis testing.
  • To demonstrate how counternull values can prevent the erroneous acceptance of null hypotheses when results are not statistically significant.

Main Methods:

  • The study defines counternull values as non-null estimand values supported by the same evidence as the null value.
  • Evidence is defined using randomization-based p-values from sharp null hypotheses in randomized experiments.
  • A counternull set, rather than a single value, is proposed for reporting.

Main Results:

  • A counternull set represents non-null effects that are statistically indistinguishable from the null effect based on the observed data.
  • Reporting counternull sets can serve a pedagogical purpose, highlighting that non-rejection of the null does not equate to acceptance.
  • Constructing counternull sets encourages deeper consideration of plausible effect sizes beyond the null.

Conclusions:

  • Reporting counternull values offers a valuable supplement to standard p-values for interpreting non-significant findings.
  • This approach enhances statistical rigor and promotes a more nuanced understanding of study results.
  • The use of counternull values can improve the accuracy of conclusions drawn from clinical and scientific research.