Emotion Recognition from Speech Signals by Mel-Spectrogram and a CNN-RNN

Area of Science:

• Computational linguistics
• Affective computing
• Machine learning for audio analysis

Background:

• Speech emotion recognition (SER) is crucial for understanding emotional well-being in health applications.
• Existing methods require robust feature extraction and temporal modeling for accurate emotion detection.

Purpose of the Study:

• To propose and evaluate a novel SER method utilizing time-frequency representations and deep neural networks.
• To enhance the accuracy of emotion detection in speech for potential health monitoring.

Main Methods:

• Speech signals are segmented and transformed into Mel-spectrograms.
• A pretrained convolutional neural network (YAMNet) extracts spectral features.
• A recurrent neural network (LSTM) models temporal dependencies between spectrograms.

Main Results:

• The proposed method achieved average accuracies of 0.711 and 0.780 on two SER datasets.
• Demonstrated relative improvement over baseline methods in emotion classification.
• Successfully identified angry, happy, sad, and neutral emotional states.

Conclusions:

• The combination of Mel-spectrograms, YAMNet, and LSTM networks offers a powerful approach for SER.
• This method shows significant potential for real-world applications in mental health and well-being monitoring.
• Further research can explore broader emotion ranges and diverse datasets.