Assessment of neural network models using prediction analysis

Examination of classification or confusion matrices in neural network models is often advocated as a way to investigate the effects of network architecture on classification rules and to understand the topography of the classification space. In addition, for model comparison purposes it is useful to be able to make statements concerning the statistical reliability of these patterns of hit and error cells. Prediction analysis provides a number of descriptive and inferential measures for the evaluation of hypotheses about patterns of hit and error cells. The present paper applied recent advances in prediction analysis to the examination of neural network classification matrices. Results of the application of prediction analysis to the generalized XOR problem and to data from classification of cardiovascular responses of human subjects were used to illustrate the determination of absolute and relative fit of omnibus models as well as the evaluation of specific hypotheses within these models.