Speech Emotion Recognition Using Convolution Neural Networks and Multi-Head Convolutional Transformer.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Speech Processing

Background:

• Speech emotion recognition (SER) is crucial for natural human-computer interaction (HCI).
• Effective extraction of emotional features from speech remains a significant challenge.
• Existing SER methods often lack advanced fusion algorithms for optimal feature representation.

Purpose of the Study:

• To develop an advanced SER system by fusing spatial and temporal feature representations.
• To leverage parallel convolutional neural networks (CNNs) and a Transformer encoder for improved emotional feature extraction.
• To enhance model robustness through data augmentation and minimize overfitting.

Main Methods:

• A novel architecture fusing parallel CNNs (for spatial features) and a Transformer encoder (for temporal features) was developed.
• The RAVDESS dataset was utilized for training and evaluation, recognizing eight distinct emotions.
• Data augmentation techniques, including Additive White Gaussian Noise (AWGN), were applied to the RAVDESS dataset.

Main Results:

• The proposed SER model achieved 82.31% accuracy on the RAVDESS dataset for eight emotions.
• Evaluation on the IEMOCAP dataset yielded 79.42% accuracy for five emotions.
• The system demonstrated superior performance compared to current state-of-the-art (SOTA) models.

Conclusions:

• The fusion of spatial and temporal features using parallel CNNs and a Transformer encoder is effective for SER.
• The developed SER system shows significant improvements in accuracy and outperforms existing SOTA models.
• The approach offers a promising direction for advancing SER in HCI applications.