Randomization tests and the unequal-N/unequal-variance problem
Related Experiment Video
Updated: Jun 21, 2026

Advancing Dyslexia Assessment in Children Through Computerized Testing
Published on: August 16, 2024
D J K Mewhort1, Mary Alexandria Kelly, Brendan T Johns
1Department of Psychology, School of Computing, Queen's University, Kingston, Ontario, Canada. mewhortd@queensu.ca
Abstract:
When both the variance and the N are unequal in a two-group design, the probability of a Type I error shifts from the nominal 5% error rate. The probability is too liberal when the small cell has the larger variance and too conservative when the large cell has the larger variance. We present an algorithm to circumvent the problem when the smaller group has the larger variance and show, by simulation, that the algorithm brings the error rate back to the nominal value without sacrificing the ability to detect true effects.
Related Concept Videos
One-Way ANOVA: Unequal Sample Sizes
One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
Friedman Two-way Analysis of Variance by Ranks
Wald-Wolfowitz Runs Test II
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...
Randomized Experiments
Simple randomization
Simple...
Test for Homogeneity
