Related Experiment Video
Updated: Jun 28, 2026

A Two-interval Forced-choice Task for Multisensory Comparisons
Published on: November 9, 2018
Aesthetics and power considerations in multiple testing--a contradiction?
1Institut für Medizinische Biometrie, Epidemiologie und Informatik, Langenbeckstrasse 1, 55101 Mainz, Germany. hommel@imbei.uni-mainz.de
Abstract:
In this paper we discuss aesthetical concepts and requirements for reasonable multiple test procedures. Aesthetical considerations lead to logical decision patterns which are conceivable and, if possible, simple to use and to communicate. Such considerations are sometimes contradictory to the ubiquitous requirement of maximizing power for a multiple test procedure. We illustrate the necessary trade-offs with several examples. We start by considering important logical properties and then discuss three different concepts of monotonicity. Afterwards we have a closer look at the recently proposed "fallback procedure" and show that it has some less appealing properties. Finally, we investigate the distribution of the numbers of significant results with respect to both expectation and variance.
Related Concept Videos
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Errors In Hypothesis Tests
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
Significance Testing: Overview
Statistical Hypothesis Testing
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...

