MAGTF-Net: Dynamic Speech Emotion Recognition with Multi-Scale Graph Attention and LLD Feature Fusion

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Traditional speech emotion recognition (SER) struggles with complex emotional nuances and dynamic context.
• Existing methods often fail to capture both local acoustic and global temporal speech structures.
• Current models lack adaptability for variable-length speech, limiting accuracy and robustness.

Purpose of the Study:

• To propose a novel SER model, MAGTF-Net (Multi-scale Attention Graph Transformer Fusion Network).
• To address limitations in capturing local/global features and handling variable-length utterances.
• To enhance the expressive capability of emotional representations for improved recognition.

Main Methods:

• Developed a log-Mel spectrogram feature extraction branch with Multi-scale Attention Graph (MAG) and Transformer encoder.
• Integrated a low-level descriptor (LLD) feature branch using a multilayer perceptron (MLP).
• Fused features from both branches and employed a label-smoothing-enhanced cross-entropy loss function.

Main Results:

• MAGTF-Net achieved weighted accuracy (WA) of 69.15% and unweighted accuracy (UA) of 70.86% on the IEMOCAP dataset.
• The model outperformed several baseline SER methods.
• Ablation studies confirmed the significant contribution of each module to performance.

Conclusions:

• The proposed MAGTF-Net effectively integrates local, global, and multi-source feature information for SER.
• The model significantly enhances the recognition of complex emotional expressions in speech.
• This work offers new theoretical and practical insights for the field of speech emotion recognition.