The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary...

Area of Science:

• Machine Learning
• Computational Statistics
• Biostatistics

Background:

• Binary classification is a key task in machine learning and statistics.
• The Area Under the Receiver Operating Characteristic Curve (ROC AUC) is a widely used metric for evaluating binary classifiers.
• However, ROC AUC has limitations, including potential overoptimistic results and insensitivity to specific performance aspects.

Purpose of the Study:

• To highlight the drawbacks of using ROC AUC as a sole performance metric for binary classification.
• To advocate for the adoption of the Matthews correlation coefficient (MCC) as a superior alternative.
• To emphasize the importance of a comprehensive evaluation of classification models.

Main Methods:

• Comparative analysis of ROC AUC and MCC.
• Examination of the components of the confusion matrix (sensitivity, specificity, precision, negative predictive value).
• Discussion of the mathematical properties and interpretation of both metrics.

Main Results:

• ROC AUC can be misleading as it doesn't account for all predictive values and can obscure poor performance in specific areas.
• MCC provides a balanced measure by considering all four fundamental rates of the confusion matrix.
• A high MCC score reliably indicates a high-performing classifier across all metrics, whereas a high ROC AUC does not guarantee this.

Conclusions:

• The Matthews correlation coefficient (MCC) is a more robust and reliable metric for evaluating binary classification performance.
• Researchers should prioritize MCC over ROC AUC to avoid overoptimistic conclusions and ensure accurate assessment of classifier efficacy.
• Adopting MCC as the standard metric will enhance the rigor and reproducibility of scientific studies employing binary classification.