Testing for normality in regression models: mistakes abound (but may not matter)

This study examines the misuse of normality tests in linear regression within ecology and biology, focusing on common misconceptions. A bibliometric review found that over 70% of ecology papers and 90% of biology papers incorrectly applied normality tests to raw data instead of model residuals. To assess the impact of this error, we simulated datasets with normal, interval, and skewed distributions across various sample and effect sizes. We compared statistical power between two approaches: testing the whole dataset for normality (incorrect) versus testing model residuals (correct) to determine whether to use a parametric (t-test) or nonparametric (Mann-Whitney U test) method. Our results showed minimal differences in statistical power between the approaches, even when normality was incorrectly tested on raw data. However, when residuals violated the normality assumption, using the Mann-Whitney U test increased statistical power by 3-4%. Overall, the study suggests that, while correctly testing residuals for normality enhances model performance, the impact of testing raw data is negligible in terms of power loss, especially with large sample sizes. The findings highlight the need for more awareness of proper statistical practices, especially in evaluating the assumptions of linear models.