Asymptotic Properties of Matthews Correlation Coefficient

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Classification evaluation is vital in statistics and machine learning, impacting critical decisions in fields like healthcare.
• The Matthews correlation coefficient (MCC) is a reliable metric, especially for imbalanced datasets.
• A research gap exists in statistical inference for MCC, leading to overreliance on point estimates.

Purpose of the Study:

• To introduce and evaluate methods for constructing asymptotic confidence intervals for the single MCC.
• To develop methods for confidence intervals of differences between MCCs in paired designs.
• To address the lack of statistical inference for MCC, enhancing reliability assessment.

Main Methods:

• Development of asymptotic confidence interval methods for single MCC.
• Construction of confidence intervals for differences in MCC for paired data.
• Simulation studies to evaluate finite-sample performance and compare methods.

Main Results:

• Evaluation of proposed confidence interval methods through simulations across various scenarios.
• Comparison of the finite-sample behavior and performance of different inference techniques.
• Demonstration of practical utility through real data analysis for binary classifier comparison.

Conclusions:

• The study provides essential statistical inference tools for the Matthews correlation coefficient.
• Findings enhance the reliability assessment of classification performance metrics.
• The research offers practical applications for comparing binary classifiers in real-world scenarios.