Related Experiment Video
Updated: Aug 13, 2026

Flypub To Study Ethanol Induced Behavioral Disinhibition and Sensitization
Published on: May 18, 2020
A comparison of the randomization test with the F test when error is skewed
1Department of Psychology, Queen's University, Kingston, Ontario, Canada. mewhortd@post.queensu.ca
Abstract:
I compared the randomization/permutation test and the F test for a two-cell comparative experiment. I varied (1) the number of observations per cell, (2) the size of the treatment effect, (3) the shape of the underlying distribution of error and, (4) for cases with skewed error, whether or not the skew was correlated with the treatment. With normal error, there was little difference between the tests. When error was skewed, by contrast, the randomization test was more sensitive than the F test, and if the amount of skew was correlated with the treatment, the advantage for the randomization test was both large and positively correlated with the treatment. I conclude that, because the randomization test was never less powerful than the F test, it should replace the F test in routine work.
Related Concept Videos
Identifying Statistically Significant Differences: The F-Test
Comparing Experimental Results: Student's t-Test
F Distribution
Friedman Two-way Analysis of Variance by Ranks
Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Randomized Experiments
Simple randomization
Simple...

