An evaluation of statistical procedures for comparing an individual's performance with that of a group of controls

The single case methodology that is widely used in cognitive neuropsychology often requires a comparison of data from a single individual (the patient) with that from a group of controls, in order to ascertain whether the patient's mean score can be viewed as significantly different from that of controls. This article reviews methods that have been used to deal with such data. Although Analysis of Variance (ANOVA) provides one possible solution of comparing group means, unequal group sizes and differences in variability between patient and controls can violate the assumptions of the test. Using Monte Carlo simulations, it was found that differences in group size and a group of N = 1 did not significantly affect the reliability of the analysis. In contrast, unacceptably high Type I errors were obtained when, in addition to unequal group sizes, there were relatively modest differences between the variance of the patient and that of the controls. We suggest that ANOVA can be used for the comparison of the mean score of an individual with that of a group of controls, but that when there is a difference in variability between the two groups, revised F criteria should be used in order to make the analysis reliable. A table of modified F values is given, which can be used for various departures from homogeneity of variance.