Combining a parallel 2D CNN with a self-attention Dilated Residual Network for CTC-based discrete speech emotion...

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Efficiently modeling long temporal contexts is a key challenge in automatic speech emotion recognition (SER).
• Recurrent Neural Networks (RNNs) are commonly used for long-term dependencies but can be computationally intensive.
• Convolutional Neural Networks (CNNs) offer potential for improved contextual information modeling in SER.

Purpose of the Study:

• To propose an efficient deep neural network architecture for discrete speech emotion recognition (SER).
• To integrate Connectionist Temporal Classification (CTC) loss for improved SER performance.
• To leverage CNN properties for enhanced modeling of temporal and frequency-based contextual information.

Main Methods:

• Developed a hybrid deep neural network architecture combining Parallel Convolutional Layers (PCN) with Squeeze-and-Excitation Network (SEnet), termed PCNSE, for feature extraction from log-Mel spectrograms.
• Employed a Self-Attention Residual Dilated Network (SADRN) with CTC as the classification block for SER.
• Utilized log-Mel spectrograms with deltas and delta-deltas as input features.

Main Results:

• The proposed hybrid PCNSE-SADRN architecture achieved a weighted accuracy (WA) of 73.1% and unweighted accuracy (UA) of 66.3% on the IEMOCAP dataset.
• The model attained a UA of 41.1% on the FAU-Aibo Emotion Corpus (FAU-AEC) dataset.
• Demonstrated the effectiveness of the novel architecture for discrete SER tasks.

Conclusions:

• The proposed hybrid deep neural network architecture offers an efficient and effective solution for discrete speech emotion recognition.
• The integration of PCNSE for feature extraction and SADRN with CTC for classification shows significant promise in SER.
• This work represents a novel application of this hybrid architecture in the field of discrete SER.