Feature and classifier-level domain adaptation in DistilHuBERT for cross-corpus speech emotion recognition.

Area of Science:

• Speech processing and machine learning
• Artificial intelligence in affective computing

Background:

• Cross-corpus speech emotion recognition (CCSER) faces challenges due to dataset variations like gender and language.
• Robust models are needed to accurately identify speaker emotions across diverse datasets.

Purpose of the Study:

• To develop effective domain adaptation strategies for CCSER using self-supervised speech representations.
• To improve the accuracy and robustness of emotion recognition models across different speech datasets.

Main Methods:

• Utilized DistilHuBERT for self-supervised speech representations.
• Proposed four Feature-level Domain Adaptation (FDA) methods, including Siamese networks and fine-tuning DistilHuBERT layers.
• Implemented Classifier-level Domain Adaptation (CDA) by updating the classifier with target dataset portions.

Main Results:

• The fourth FDA method, combined with CDA, yielded the highest accuracy.
• Achieved 92.01% accuracy on the EMODB dataset using ShEMO as the source dataset.
• Demonstrated the effectiveness of proposed domain adaptation techniques in CCSER.

Conclusions:

• The proposed FDA and CDA methods significantly improve CCSER performance.
• Fine-tuning both CNN and transformer layers of DistilHuBERT (fourth FDA method) combined with CDA is a promising approach for robust emotion recognition.
• This research contributes to more accurate and reliable emotion identification systems across varied speech data.