Review of the two sample t tests

The t test is a valuable statistical manipulation of moderate strength to determine whether a significant difference exists between the means of two groups, either paired or unpaired. Generally, the larger the t value, the greater chance of its statistical significance. Sample size also will influence the point at which t becomes significant, that is, the larger the size of n, the smaller the t required to become significant. As the number of degrees of freedom becomes larger, a smaller t value is sufficient to reject the null hypothesis, and as variability increases, the true chance for significant difference decreases. It should be noted that a common error in the use of the t test occurs with excessive repetition of the test on the same dataset. A resultant type 1 error will be introduced that incorrectly concludes that significant differences have been demonstrated when, in fact, they have resulted not from significance but from repeated statistical application. In other words, if a .05 level of significance is used repeatedly on a dataset, the investigator is assuming that there is a 1 in 20 chance of finding a significant difference when there is no true difference between variables. It becomes obvious that if 20 repeated applications of the t test were performed, one would expect to find one significant difference by chance alone when no true difference exists. If more than 10 applications of the t test are being conducted on the same dataset, the investigator should lower the level of significance (.01) on each individual t test of the entire set so fewer null hypotheses are rejected. t Tests numbering 40 to 50 should have an alpha level of .005. An alternative approach would be to consult a statistician and use multivariate statistical procedures.