Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Multiple Comparison Tests

Multiple Comparison Tests

Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...

Bonferroni Test

Bonferroni Test

The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...

Errors In Hypothesis Tests

Errors In Hypothesis Tests

When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.

Accuracy and Errors in Hypothesis Testing

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...

Significance Testing: Overview

Significance Testing: Overview

Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...

Statistical Hypothesis Testing

Statistical Hypothesis Testing

Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

The CRT-Estimands Framework for Cluster Randomized Trials.

JAMA·2026

Same author

CRT-Estimands Framework: consensus based extension of the ICH E9(R1) addendum for cluster randomised trials.

BMJ (Clinical research ed.)·2026

Same author

STrategies for developing REseArch Methods guidance (STREAM): Protocol.

Journal of clinical epidemiology·2026

Same author

Development of a consensus extension of the estimands framework for cluster randomised trials (CRT-estimands): results from an international Delphi study.

Trials·2025

Same author

From Data to Knowledge. Advancing Life Sciences: Editorial for the CEN2023 Special Issue.

Biometrical journal. Biometrische Zeitschrift·2025

Same author

Minimum Area Confidence Set Optimality for Simultaneous Confidence Bands for Percentiles With Applications to Drug Shelf-Life Estimation.

Statistics in medicine·2025

Search research articles

Related Experiment Video

Updated: Jun 28, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

Aesthetics and power considerations in multiple testing--a contradiction?

Gerhard Hommel¹, Frank Bretz

¹Institut für Medizinische Biometrie, Epidemiologie und Informatik, Langenbeckstrasse 1, 55101 Mainz, Germany. hommel@imbei.uni-mainz.de

Biometrical Journal. Biometrische Zeitschrift

|October 22, 2008

Summary

This study explores logical and aesthetic criteria for multiple testing procedures, balancing simplicity with statistical power. It examines monotonicity and the fallback procedure, analyzing the distribution of significant results.

More Related Videos

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Published on: September 27, 2019

Related Experiment Videos

Last Updated: Jun 28, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons

A Two-interval Forced-choice Task for Multisensory Comparisons

Published on: November 9, 2018

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Applying an eMASS Customization Program as a Research Tool to Evaluate Consumer Benefits

Published on: September 27, 2019

Area of Science:

Statistics
Multiple Hypothesis Testing

Background:

Traditional multiple testing procedures often prioritize statistical power.
Aesthetic considerations, such as simplicity and interpretability, are crucial for practical application but can conflict with maximizing power.

Purpose of the Study:

To investigate aesthetic requirements for multiple testing procedures.
To analyze the trade-offs between aesthetic principles and statistical power.
To evaluate the properties of specific multiple testing procedures, including monotonicity and the fallback procedure.

Main Methods:

Discussion of logical properties and decision patterns in multiple testing.
Examination of three distinct concepts of monotonicity.
Analysis of the recently proposed "fallback procedure".
Investigation of the expectation and variance of significant results.

Main Results:

Aesthetic considerations can lead to logical decision patterns that are simpler to use and communicate.
There are inherent trade-offs between aesthetic goals and maximizing the power of multiple testing procedures.
The fallback procedure exhibits less desirable properties.
The distribution of significant results, in terms of expectation and variance, was analyzed.

Conclusions:

Reasonable multiple testing procedures should incorporate aesthetic and logical considerations alongside statistical power.
Simplicity and communicability are important factors in procedure design.
The properties of specific procedures like the fallback procedure require careful evaluation.