Enhanced graph attention network by integrating Long Short-Term Memory for artificial emotion representation in...

Area of Science:

• Artificial Intelligence
• Affective Computing
• Human-Computer Interaction

Background:

• Emotion recognition from multi-modal data is challenging due to complex feature relationships.
• Existing methods struggle to integrate textual, audio, and visual emotional cues effectively.

Purpose of the Study:

• To develop a hybrid AI model for robust emotion recognition across text-only and multi-modal data.
• To improve the integration of semantic and temporal features for accurate emotion detection.

Main Methods:

• Utilized Enhanced Graph Attention Networks (E-GAT) to capture structural dependencies from text embeddings.
• Employed Bidirectional Long Short-Term Memory (Bi-LSTM) to model temporal dynamics in sequential data.
• Integrated E-GAT and Bi-LSTM for a unified emotion recognition framework.

Main Results:

• Achieved state-of-the-art performance on the SemEval-2018 (text-only) dataset with 58.5% accuracy and 68.7% F1-score.
• Demonstrated high accuracy on multi-modal datasets: 78.9% on RAVDESS and 82.3% on CMU-MOSEI.
• Showcased robust cross-modal generalization capabilities.

Conclusions:

• The proposed hybrid E-GAT and Bi-LSTM model effectively addresses challenges in multi-modal emotion recognition.
• This unified framework advances emotion recognition for applications in HCI and mental health monitoring.
• The model offers significant improvements over baseline approaches for diverse emotional data types.