Performance of five two-sample location tests for skewed distributions with unequal variances

Tests for comparing the locations of two independent populations are associated with different null hypotheses, but results are often interpreted as evidence for or against equality of means or medians. We examine the appropriateness of this practice by investigating the performance of five frequently used tests: the two-sample T test, the Welch U test, the Yuen-Welch test, the Wilcoxon-Mann-Whitney test, and the Brunner-Munzel test. Under combined violations of normality and variance homogeneity, the true significance level and power of the tests depend on a complex interplay of several factors. In a wide ranging simulation study, we consider scenarios differing in skewness, skewness heterogeneity, variance heterogeneity, sample size, and sample size ratio. We find that small differences in distribution properties can alter test performance markedly, thus confounding the effort to present simple test recommendations. Instead, we provide detailed recommendations in Appendix A. The Welch U test is recommended most frequently, but cannot be considered an omnibus test for this problem.