Modeling Hierarchical Uncertainty for Multimodal Emotion Recognition in Conversation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Human-Computer Interaction

Background:

• Estimating uncertainty in AI is vital for reliable human-AI interaction, especially in critical scenarios.
• Existing emotion recognition in conversation (ERC) systems lack uncertainty estimation capabilities.
• This gap hinders the deployment of dependable AI agents in sensitive applications.

Purpose of the Study:

• To introduce HU-Dialogue, a novel framework for modeling hierarchical uncertainty in ERC.
• To enhance the reliability of AI agents by quantifying prediction uncertainty.
• To facilitate human-in-the-loop solutions in conversational AI.

Main Methods:

• HU-Dialogue employs a regularization scheme perturbing contextual attention weights with source-adaptive noises for context-level uncertainty.
• Bayesian deep learning is adapted to a capsule-based prediction layer for modality-level uncertainty.
• A weight-sharing triplet structure with conditional layer normalization is utilized to detect inter-modal invariance and equivariance.

Main Results:

• The proposed HU-Dialogue model demonstrates superior performance compared to state-of-the-art methods.
• Empirical analysis on three popular multimodal ERC datasets validates the model's effectiveness.
• The hierarchical uncertainty modeling significantly improves ERC accuracy and reliability.

Conclusions:

• HU-Dialogue successfully models hierarchical uncertainty in conversational emotion recognition.
• The framework enhances AI reliability and is suitable for critical applications requiring human oversight.
• This work advances the field of explainable and trustworthy AI in human-computer interaction.