On the use of post-hoc tests in environmental and biological sciences: A critical review

Post-hoc comparison procedures are commonly used to determine which group means differ after a significant analysis of variance (ANOVA). Several post-hoc tests have been proposed, but their use requires certain assumptions to be met, such as normality, equality of variance, and balanced group size. This review examined the statistical literature on post-hoc tests and their use in the environmental and biological sciences. Through this review, we found that post-hoc tests are effective but often inadequately used in these sciences. We conducted a search of reputed search engines to identify articles in which post-hoc tests were used and found ten post-hoc tests used in the environmental and biological literature. Tukey HSD (30.04%), Duncan's (25.41%) and Fisher's LSD (18.15%) were the most commonly used post-hoc tests over the past 20 years, whereas the Games-Howell (1.13%), Holm-Bonferroni (1.25%), and Scheffe's tests (2.25%) were the least used. The choice of post-hoc test depended on the statistical method used prior. In addition, the assumptions of applying post-hoc tests were not always verified. In fact, the normality condition was mostly only checked in the cases of Tukey HSD, Duncan's, and Fischer's LSD tests, and equality of variance was often met for the Tukey HSD, Duncan's, Fischer's LSD, and Bonferroni tests. This review opens a new avenue for comparing post-hoc test performance in ANOVA using linear or generalised mixed effect models.