Beyond Global Metrics: A Fairness Analysis for Interpretable Voice Disorder Detection Systems.

Area of Science:

• Biomedical Engineering
• Speech Science
• Artificial Intelligence in Healthcare

Background:

• Automatic Voice Disorder Detection (AVDD) systems are crucial for diagnosing speech impairments.
• Existing AVDD systems often exhibit performance disparities across different demographic groups.
• Global performance metrics may mask underlying biases in AVDD system accuracy.

Purpose of the Study:

• To investigate and quantify demographic-dependent biases in AVDD systems.
• To evaluate the effectiveness of group-specific calibration strategies for enhancing AVDD reliability.
• To identify factors contributing to performance differences across age and gender cohorts.

Main Methods:

• Analysis of an AVDD system using voice disorder datasets with demographic metadata.
• Evaluation of system performance across gender and age-based cohorts using normalized costs and cross-entropy.
• Application of group-specific calibration techniques to mitigate miscalibration.

Main Results:

• Significant performance disparities were observed across demographic groups, despite strong overall metrics.
• The AVDD system exhibited biases, misclassifying older healthy speakers and younger disordered speakers.
• Group-specific calibration improved posterior probability estimation and reduced overconfidence.

Conclusions:

• Global performance metrics are insufficient for a fair evaluation of AVDD systems.
• Demographic-specific analysis and calibration are essential for reliable voice disorder detection.
• The findings provide a framework for improving bias mitigation in biomedical classification tasks.