Further evaluating the conditional decision rule for comparing two independent means

Many books on statistical methods advocate a 'conditional decision rule' when comparing two independent group means. This rule states that the decision as to whether to use a 'pooled variance' test that assumes equality of variance or a 'separate variance' Welch t test that does not should be based on the outcome of a variance equality test. In this paper, we empirically examine the Type I error rate of the conditional decision rule using four variance equality tests and compare this error rate to the unconditional use of either of the t tests (i.e. irrespective of the outcome of a variance homogeneity test) as well as several resampling-based alternatives when sampling from 49 distributions varying in skewness and kurtosis. Several unconditional tests including the separate variance test performed as well as or better than the conditional decision rule across situations. These results extend and generalize the findings of previous researchers who have argued that the conditional decision rule should be abandoned.