Hypothesis testing procedure for binary and multi-class F₁ -scores in the paired design

Area of Science:

• Medical Diagnostics
• Statistical Methods
• Machine Learning Evaluation

Background:

• Medical tests are crucial for diagnosis, screening, and risk prediction.
• Performance metrics like sensitivity, specificity, and predictive values are standard for binary tests.
• The F1-score, a harmonic mean of precision and recall, is increasingly used, with micro- and macro-averaged versions for multi-class problems.

Purpose of the Study:

• To develop a hypothesis testing procedure for comparing two F1-scores.
• To address the gap in statistical inference methodologies for F1-scores, particularly in paired study designs.
• To provide a robust statistical framework for evaluating multi-class medical classification models.

Main Methods:

• Developing a hypothesis testing procedure for comparing two F1-scores.
• Utilizing the large sample multivariate central limit theorem.
• Applying methods to paired study designs for medical test performance evaluation.

Main Results:

• A novel hypothesis testing procedure for comparing F1-scores in paired studies has been developed.
• The methodology is grounded in the large sample multivariate central limit theorem, ensuring statistical validity.
• This contributes to advancing statistical inference for machine learning model evaluation in medicine.

Conclusions:

• The developed hypothesis testing procedure offers a statistically sound method for comparing F1-scores in paired medical study designs.
• This research enhances the statistical toolkit for evaluating the performance of medical diagnostic tests.
• Further methodological development in hypothesis testing for F1-scores is crucial for advancing medical data science.