Preference-Driven Classification Measure.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• Assessing machine learning classification quality is complex, with existing measures often limited to binary classification.
• Multi-class classification presents challenges, necessitating adaptable evaluation metrics.
• Current methods may oversimplify quality assessment for datasets with numerous decision classes.

Purpose of the Study:

• To propose a novel preference-driven measure (p-d) for classifier quality assessment applicable to any number of classes.
• To enable the incorporation of relative class importance into the evaluation process.
• To demonstrate a flexible approach for adapting classifier assessment to specific analytical needs via preference vectors.

Main Methods:

• Development of a new preference-driven measure (p-d) for classification quality.
• Application of the p-d measure to a two-class decision problem for visualization.
• Testing the p-d measure on real-world multi-class datasets.
• Demonstration of user-specific preference adjustment for classifier assessment.

Main Results:

• The preference-driven measure effectively evaluates classifier quality across multi-class scenarios.
• The p-d measure allows for tailored assessments based on user-defined class priorities.
• Comparative analysis shows the p-d measure can identify superior classifiers based on specific preferences, outperforming classical metrics.

Conclusions:

• The proposed preference-driven measure offers a robust and adaptable solution for evaluating multi-class classification.
• This approach enhances classifier selection by aligning evaluation with specific problem contexts and user priorities.
• The p-d measure provides a more insightful alternative to traditional classification quality assessment methods.