A Hybrid Time-Distributed Deep Neural Architecture for Speech Emotion Recognition.

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence
• Signal Processing

Background:

• Speech emotion recognition (SER) is crucial for adaptive human-machine interaction.
• Voice analysis offers a rich, non-invasive method for assessing user emotional states.
• Current systems require improved accuracy and robustness in emotion detection.

Purpose of the Study:

• To propose a novel hybrid deep learning architecture for SER.
• To enhance SER performance using Mel-frequency log-power spectrograms (MFLPSs) and data augmentation.
• To validate the proposed model against state-of-the-art methods on a challenging benchmark dataset.

Main Methods:

• A hybrid model combining a time-distributed convolutional neural network (TD-CNN) and a long short-term memory (LSTM) network was developed.
• Mel-frequency log-power spectrograms (MFLPSs) were extracted and processed using a sliding window for the TD-CNN input.
• Innovative image data augmentation techniques were applied to the MFLPS representation to mitigate overfitting.

Main Results:

• The hybrid TD-CNN-LSTM model achieved an average recognition accuracy of 73.98% on a widely used SER benchmark database.
• A permutation test confirmed the statistical significance of the results, distinguishing them from random classification.
• The proposed architecture demonstrated superior performance compared to existing deep learning and conventional machine learning techniques.

Conclusions:

• The developed hybrid TD-CNN-LSTM model represents a significant advancement in speech emotion recognition.
• The MFLPS representation and associated augmentation techniques offer effective strategies for improving SER accuracy.
• This research provides a robust and high-performing solution for real-world applications requiring user emotion assessment.