Evaluation of neural network performance by receiver operating characteristic (ROC) analysis: examples from the...

A need exists for an unbiased measure of the accuracy of feed-forward neural networks used for classification. Receiver operating characteristic (ROC) analysis is suited for this measure, and has been used to assess the performance of several different network weights. The area under an ROC and its standard error were used to compare different network weight sets, and to follow the performance of a network during the course of training. The ROC is not sensitive to the prior probabilities of examples in the testing set nor to the system's decision bias. The area under an ROC curve is a readily understood measure, and should be used to evaluate neural networks and to report results of learning experiments. Examples are provided from experiments with data from the biotechnology domain.