Multitask and Transfer Learning Approach for Joint Classification and Severity Estimation of Dysphonia

Area of Science:

• Speech analysis
• Biomedical engineering
• Machine learning

Background:

• Speech analysis offers insights into health, emotions, and identity.
• Voice disorders can impact vocal organs, leading to communication difficulties.
• Previous research often focused on single speech analysis tasks.

Purpose of the Study:

• To develop a multitask learning (MTL) approach for simultaneous dysphonia classification and severity estimation.
• To leverage shared representations and computer vision models for improved voice analysis.
• To investigate the impact of auxiliary tasks, like speaker sex classification, on performance.

Main Methods:

• Combined transfer learning and multitask learning (MTL) for voice analysis.
• Adapted five computer vision models to support MTL.
• Conducted binary and multiclass dysphonia classification with speaker sex as an auxiliary task.

Main Results:

• MTL demonstrated superior performance across classification metrics compared to single-task learning (STL).
• Achieved F1 scores of 93% (MTL) vs. 90% (STL) for classification.
• MTL accuracy reached 77% compared to STL's 73.2%, with comparable severity estimation.

Conclusions:

• MTL enhances the accuracy of voice disorder classification and severity estimation.
• The proposed method aids clinicians in understanding patient conditions and monitoring progress.
• Integrating classification and severity estimation improves vocal quality monitoring and treatment assessment.