Speech emotion recognition based on transfer learning from the FaceNet framework.

Area of Science:

• Computer Science
• Artificial Intelligence
• Signal Processing

Background:

• Human-computer interaction increasingly relies on emotional understanding.
• FaceNet, a successful face recognition model, offers robust feature extraction capabilities.
• Adapting advanced models like FaceNet for speech emotion recognition (SER) is a promising research direction.

Purpose of the Study:

• To adapt and improve the FaceNet model for accurate speech emotion recognition.
• To investigate the effectiveness of pretraining and fine-tuning strategies for SER using FaceNet.
• To evaluate the proposed approach against state-of-the-art methods on benchmark datasets.

Main Methods:

• Speech signals were segmented and transformed into waveform diagrams and spectrograms.
• The FaceNet model was adapted for end-to-end training on both signal representations.
• Pretraining on the CASIA dataset and fine-tuning on the IEMOCAP dataset were employed, leveraging transfer learning.
• Cross-training experiments were conducted for comprehensive evaluation.

Main Results:

• Pretraining FaceNet on spectrograms demonstrated effectiveness.
• The adapted model achieved 90% accuracy on the CASIA dataset and 68.96% on the IEMOCAP dataset.
• The proposed approach outperformed state-of-the-art single-modal methods on the IEMOCAP dataset.

Conclusions:

• The adapted FaceNet model shows significant potential for advancing speech emotion recognition.
• Transfer learning, particularly pretraining on high-accuracy datasets like CASIA, is beneficial for SER.
• The method offers a competitive single-modal approach for SER, especially on challenging datasets like IEMOCAP.