Confidence interval for micro-averaged F ₁ and macro-averaged F ₁ scores

Area of Science:

• Statistics
• Computer Science
• Machine Learning

Background:

• Binary classification metrics like sensitivity and specificity are common in medicine.
• Precision and recall are standard for classifier evaluation in computer science.
• The F1 score, a harmonic mean of precision and recall, is crucial for imbalanced datasets but lacks multi-class extensions.

Purpose of the Study:

• To address the lack of statistical inference methods for F1 scores in multi-class classification.
• To propose novel methods for estimating F1 scores and their confidence intervals.
• To enhance the evaluation of classifier performance in complex scenarios.

Main Methods:

• Development of statistical methods based on the large sample multivariate central limit theorem.
• Application to estimating three types of F1 scores in multi-class classification.
• Focus on providing confidence intervals for F1 score estimates.

Main Results:

• Proposed methods enable statistical inference for multi-class F1 scores.
• Confidence intervals can be estimated for various F1 score types.
• The methods are grounded in established statistical theory.

Conclusions:

• The developed methods fill a critical gap in multi-class classification evaluation.
• Provides a robust framework for understanding F1 score uncertainty.
• Facilitates more reliable performance assessment of classifiers in diverse applications.