DialogueLLM: Context and emotion knowledge-tuned large language models for emotion recognition in conversations

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computational Linguistics

Background:

• Large language models (LLMs) demonstrate strong performance in natural language generation but lack specialized emotion understanding capabilities.
• Current LLMs often fail to leverage multi-modal information, limiting their precision in emotion recognition tasks.
• Existing LLMs may provide suboptimal results for emotion recognition due to a lack of context-awareness and emotion-specific tuning.

Purpose of the Study:

• To develop a novel large language model, DialogueLLM, specifically tuned for context-aware emotion recognition.
• To address the limitations of general-purpose LLMs in accurately understanding and recognizing emotions in dialogues.
• To create a robust framework for emotion recognition by modeling it as a text generation task.

Main Methods:

• DialogueLLM was developed by fine-tuning foundation large language models with context and emotion knowledge.
• A large-scale dataset of over 24,000 utterances was created to serve as a knowledge corpus for training emotional LLMs.
• ERNIE Bot was prompted to generate textual descriptions of videos, integrating multi-modal information.

Main Results:

• DialogueLLM significantly outperformed 15 state-of-the-art baselines and 3 state-of-the-art LLMs on three benchmarking datasets.
• An emotion intelligence test showed DialogueLLM achieved a score of 109, surpassing 72% of human performance.
• DialogueLLM-7B demonstrated reproducibility, achievable with LoRA on a 40GB A100 GPU in 5 hours.

Conclusions:

• DialogueLLM represents a significant advancement in emotion recognition through specialized LLM fine-tuning.
• The context-aware nature of DialogueLLM enables accurate capture of emotional dynamics within dialogues.
• DialogueLLM offers a promising, efficient, and high-performing solution for multi-modal emotion recognition tasks.