DSTCNet: Deep Spectro-Temporal-Channel Attention Network for Speech Emotion Recognition

Area of Science:

• Artificial Intelligence
• Machine Learning
• Signal Processing

Background:

• Speech emotion recognition (SER) is crucial for improving human-computer interaction.
• Current deep learning methods, particularly CNNs, are widely used but have limitations in modeling feature associations.
• CNNs applied to spectrograms often fail to explicitly capture spectral, temporal, and channel-wise feature relevance.

Purpose of the Study:

• To propose a novel deep spectro-temporal-channel network (DSTCNet) for enhanced speech emotion recognition.
• To improve the representational learning capabilities for distinguishing emotions in speech.
• To address the limitations of traditional CNNs in capturing feature associations.

Main Methods:

• Developed a Deep Spectro-Temporal-Channel Network (DSTCNet) integrating spectro-temporal-channel (STC) attention modules into a CNN architecture.
• Proposed an STC module to infer a 3-D attention map across time, frequency, and channel dimensions.
• Trained and evaluated the DSTCNet on the Berlin emotional database (EmoDB) and IEMOCAP database.

Main Results:

• The proposed DSTCNet demonstrated superior performance compared to traditional CNN-based methods.
• DSTCNet outperformed several existing state-of-the-art approaches in speech emotion recognition.
• The STC attention mechanism effectively focused on crucial speech features across different dimensions.

Conclusions:

• The DSTCNet significantly improves speech emotion recognition by effectively modeling spectro-temporal-channel feature associations.
• The proposed STC attention module enhances the ability of deep learning models to learn discriminative emotional representations from speech.
• DSTCNet offers a promising advancement for more interactive and intuitive human-computer systems.