End-to-end emotional speech recognition using acoustic model adaptation based on knowledge distillation

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Traditional hidden Markov model-deep neural network (HMM-DNN) approaches struggle with abnormal speech, particularly emotional speech recognition.
• End-to-end models offer better performance but still exhibit limitations with emotional speech variations.
• Collecting sufficient emotional speech data for each specific emotion is a significant challenge for building specialized acoustic models.

Purpose of the Study:

• To propose a novel method for enhancing emotional speech recognition performance.
• To adapt acoustic models for emotional speech using knowledge distillation.
• To address the data scarcity issue in training emotion-specific speech recognition models.

Main Methods:

• Utilizing knowledge distillation, a technique typically used for model compression, for model adaptation to emotional speech.
• Developing a 'teacher' model with numerous parameters trained on normal speech data.
• Constructing a 'student' model with fewer parameters trained on a small dataset of emotional speech (adaptation data).

Main Results:

• The student model demonstrated consistent recognition performance irrespective of parameter count.
• The teacher model's performance significantly degraded with reduced parameters, showing a ~10% increase in word error rate.
• The student model effectively captured emotional characteristics, proving suitable for emotional speech recognition.

Conclusions:

• Knowledge distillation enables effective adaptation of acoustic models for emotional speech recognition using limited data.
• The proposed student model approach is robust to parameter reduction, unlike traditional models.
• This method provides a viable solution for improving emotional speech recognition without requiring large, emotion-specific datasets.