End-to-end Acoustic-linguistic Emotion and Intent Recognition Enhanced by Semi-supervised Learning.

Area of Science:

• Speech processing
• Machine learning
• Human-computer interaction

Background:

• Emotion and intent recognition from speech are crucial for human-computer interaction.
• Large volumes of speech data from social media and chatbots present annotation challenges.
• Manual annotation is costly, hindering the training of effective machine learning models.

Purpose of the Study:

• To apply semi-supervised learning for speech emotion and intent recognition.
• To leverage large-scale unlabelled data alongside limited labelled data.
• To compare fix-match and full-match semi-supervised learning approaches.

Main Methods:

• Training end-to-end acoustic and linguistic models.
• Employing multi-task learning for both emotion and intent recognition.
• Utilizing semi-supervised learning (fix-match and full-match) with labelled and unlabelled data.

Main Results:

• Semi-supervised learning significantly improves model performance for speech emotion and intent recognition.
• Both acoustic and text data benefit from the proposed semi-supervised approaches.
• Late fusion of models achieved superior performance over acoustic and text baselines.

Conclusions:

• Semi-supervised learning is effective for enhancing speech emotion and intent recognition.
• The integration of unlabelled data addresses annotation cost challenges.
• The proposed methods offer improved accuracy in human-computer interaction systems.