A statistical comparison between Matthews correlation coefficient (MCC), prevalence threshold, and Fowlkes-Mallows...

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Assessing binary classifications is crucial in scientific research, but a consensus on a single summary statistic for confusion matrices remains elusive.
• Previous work highlighted the advantages of the Matthews correlation coefficient (MCC) over various other metrics like accuracy, F1 score, and Cohen's kappa.
• The need for robust metrics is evident across diverse scientific disciplines.

Purpose of the Study:

• To compare the Matthews correlation coefficient (MCC) with prevalence threshold (PT) and Fowlkes-Mallows index.
• To investigate the interrelationships between these three statistical metrics.
• To determine the conditions under which MCC offers superior information for binary classification evaluation.

Main Methods:

• Comparative analysis of statistical metrics for binary classification.
• Investigation of mutual relations among Matthews correlation coefficient (MCC), prevalence threshold (PT), and Fowlkes-Mallows index.
• Evaluation using relevant use cases from scientific research.

Main Results:

• The Matthews correlation coefficient (MCC) was compared against prevalence threshold (PT) and Fowlkes-Mallows index.
• Mutual relations among the three metrics were analyzed.
• Results indicate MCC can be more informative than PT and Fowlkes-Mallows index when positive and negative data elements hold equal importance.

Conclusions:

• The Matthews correlation coefficient (MCC) demonstrates significant advantages in evaluating binary classifications, particularly when class balance is considered.
• MCC provides more comprehensive information compared to prevalence threshold (PT) and Fowlkes-Mallows index under specific conditions of equal data element importance.
• The findings support the broader adoption of MCC as a preferred metric for binary classification tasks in scientific research.