Evaluation of neural network performance by receiver operating characteristic (ROC) analysis: examples from the...

Area of Science:

• Machine Learning
• Biotechnology
• Computational Biology

Background:

• Feed-forward neural networks are widely used for classification tasks.
• Assessing the accuracy of these networks requires unbiased performance measures.
• Existing methods may be sensitive to dataset biases and decision thresholds.

Purpose of the Study:

• To introduce Receiver Operating Characteristic (ROC) analysis as an unbiased measure for neural network classification accuracy.
• To demonstrate the utility of ROC analysis for comparing network weights and monitoring training progress.
• To highlight ROC analysis's independence from prior probabilities and decision bias.

Main Methods:

• Applied Receiver Operating Characteristic (ROC) analysis to evaluate feed-forward neural networks.
• Utilized the area under the ROC curve (AUC) and its standard error for performance comparison.
• Conducted experiments using data from the biotechnology domain.

Main Results:

• ROC analysis provides an unbiased assessment of classification accuracy for neural networks.
• The area under the ROC curve (AUC) effectively compares different network weight sets.
• ROC analysis can track network performance during the training process.
• ROC analysis is not affected by prior probabilities or decision bias.

Conclusions:

• Receiver Operating Characteristic (ROC) analysis is a robust and readily understood metric for evaluating neural network performance.
• The area under the ROC curve (AUC) should be adopted for reporting results in machine learning and biotechnology research.
• ROC analysis offers a reliable method for unbiased performance evaluation in classification tasks.