Accurate semi-supervised automatic speech recognition for ordinary and characterized speeches via...

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Automatic Speech Recognition (ASR) systems are crucial for applications like translation and transcription.
• Current ASR models are specialized for either ordinary or characterized speech.
• Semi-supervised learning is gaining traction due to the high cost and scarcity of labeled speech data.

Purpose of the Study:

• To develop accurate semi-supervised ASR models for both ordinary and characterized speech.
• To address the limitations of pseudo-labeling in previous semi-supervised ASR approaches.
• To improve ASR performance, especially for characterized speech with limited data.

Main Methods:

• Proposed MOCA (Multi-hypotheses-based Curriculum learning for semi-supervised Asr) for ordinary speech.
• Developed MOCA-S for characterized speech, leveraging data from other speech types.
• MOCA and MOCA-S generate multiple hypotheses per instance to mitigate pseudo-label dependency.
• MOCA-S dynamically adjusts pseudo-labeling based on trait relevance.

Main Results:

• MOCA and MOCA-S demonstrated significant improvements in ASR accuracy compared to existing models.
• The multi-hypotheses approach effectively reduced reliance on inaccurate pseudo labels.
• MOCA-S successfully utilized data from other speech traits to enhance characterized speech recognition.

Conclusions:

• The proposed MOCA and MOCA-S frameworks offer a robust solution for semi-supervised ASR.
• These methods enhance transcription accuracy for diverse speech types, including those with unique characteristics.
• The study highlights the effectiveness of multi-hypotheses generation and cross-trait data utilization in ASR.