An innovative 3D attention mechanism for multi-label emotion classification

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Current multi-label emotion classification often relies on pre-trained models and attention mechanisms.
• Existing research primarily focuses on altering attention network structures or using larger models, neglecting the enhancement of attention's learning capabilities.
• The GoEmotions dataset, with 28 emotion categories, presents a complex challenge for emotion classification.

Purpose of the Study:

• To introduce a novel attention mechanism, Commander Attention, designed to improve multi-label emotion classification.
• To enhance the learning capabilities of attention mechanisms by incorporating emotional polarity and intensity.
• To evaluate the effectiveness of Commander Attention on a complex, multi-label emotion dataset.

Main Methods:

• Developed distinct attention layers for each emotion label, forming a 3D cube structure.
• Incorporated emotional polarity and intensity as a 'Commander' feature to adjust attention weights.
• Employed multi-head attention to connect attention planes and utilized multi-task learning for independent emotion predictions.
• Integrated Commander Attention with the XLNet pre-trained model for fine-tuning.

Main Results:

• Commander Attention (3-CA) significantly outperforms the original method in classifying all emotion categories.
• The proposed method achieves a minimum improvement in over 85.7% of emotion categories compared to state-of-the-art methods.
• The novel 3D attention space and indicative features effectively guide attention learning.

Conclusions:

• Commander Attention represents a significant advancement in multi-label emotion classification by enhancing attention mechanisms.
• The incorporation of emotional polarity and intensity, along with a 3D attention space, leads to superior performance on complex emotion datasets.
• The approach offers a promising direction for future research in emotion recognition and sentiment analysis.