Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing
• Computer Vision
• Speech Processing

Background:

• Empathetic machines require accurate emotion recognition in conversations.
• Current methods often overlook inter-speaker dependencies, limiting performance.
• Recognizing utterance-level emotions in dyadic conversations is a key challenge.

Purpose of the Study:

• To develop a novel deep neural framework for recognizing utterance-level emotions in dyadic conversational videos.
• To address the limitation of existing methods by incorporating inter-speaker dependency relations.
• To improve the accuracy of emotion recognition in conversations.

Main Methods:

• Proposed a deep neural framework termed conversational memory network (CMN).
• Employed a multimodal approach using audio, visual, and textual features.
• Utilized gated recurrent units (GRUs) to model speaker memories and attention-based hops to capture inter-speaker dependencies.

Main Results:

• The conversational memory network achieved a 3-4% accuracy improvement over state-of-the-art methods.
• Demonstrated the effectiveness of leveraging contextual information and inter-speaker dependencies.
• Validated the multimodal approach for enhanced emotion recognition.

Conclusions:

• The proposed conversational memory network effectively models inter-speaker dependencies for improved emotion recognition.
• Multimodal features combined with memory networks offer a promising direction for empathetic AI.
• This work advances the state of the art in conversational emotion recognition.