Contingency Space: A Semimetric Space for Classification Evaluation.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Current machine learning evaluation metrics suffer from limitations like one-dimensionality, lack of context, and binary restrictions.
• These limitations hinder effective model performance assessment and comparison.

Purpose of the Study:

• To propose a novel framework, Contingency Space, for a generic representation of performance evaluation metrics.
• To address the limitations of existing metrics and enable visual comparison and analysis.
• To introduce new concepts for quantifying imbalance sensitivity and analyzing training processes.

Main Methods:

• Developed Contingency Space, a bounded semimetric space, to represent any performance evaluation metric.
• Introduced Imbalance Sensitivity to quantify a metric's sensitivity to class imbalance.
• Defined Learning Path for qualitative and quantitative analysis of model training.
• Proposed Tau, a new cost-sensitive and Imbalance Agnostic metric.

Main Results:

• Contingency Space visually compares different evaluation metrics using surfaces.
• Imbalance Sensitivity quantifies metric skew-sensitivity.
• Learning Path provides insights into the training process.
• Tau metric offers improved performance evaluation in cost-sensitive and imbalanced scenarios.
• Demonstrated applicability to multi-class problems.

Conclusions:

• Contingency Space offers a unified and versatile approach to machine learning metric evaluation.
• The proposed methods enhance the understanding and comparison of evaluation metrics.
• This framework provides practical tools for analyzing model performance and training dynamics.