Assessing Predictive Accuracy in Discriminant Analysis

The estimation of probabilities of correct classification is a primary concern in predictive discriminant analysis. Three such probabilities are: (a) the optimal hit rate, that obtained when the classification rule is based on known parameters; (b) the actual hit rate, that obtained by applying a rule based on a particular sample to future samples; and (c) the expected actual hit rate. Methods of estimating these hit rates include formulas (in the two-group case), resubstitution, and external analyses. The methods are tentatively compared via Monte Carlo sampling from two real data sets.