相关实验视频
Updated: Sep 11, 2025

13:51
Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
20.1K
跨模式封闭功能增强用于多模式情绪识别在对话中
Shiyun Zhao1, Jinchang Ren2,3, Xiaojuan Zhou1
1Department of Economics and Management, Suzhou Chien-Shiung Institute of Technology, Suzhou, 215411, China.
Scientific reports
|August 16, 2025
概括
这项研究引入了一种用于对话中的情绪识别 (ERC) 的新型跨模态封闭注意力机制. 该方法有效地融合了视觉,听觉和文本数据,提高了 empathetic AI 系统的准确性和稳定性.
科学领域:
- 人工智能的人工智能
- 人与计算机的交互
- 计算语言学 计算语言学
背景情况:
- 在对话中的情感识别 (ERC) 对像客户服务和健康监控这样的同情的人工智能应用至关重要.
- 目前的多式联运ERC方法在特征提取,使用不足的互补数据和跨式联运冗余等方面扎.
- 有效的多式情绪识别需要各种感官输入的强大的融合.
研究的目的:
- 开发一种先进的多式联络式情绪识别系统,使用一种新的跨式联络式封闭注意力机制.
- 通过有效地融合视觉,听觉和文本特征来提高情感识别的准确性和稳定性.
- 为了应对特征提取,互补特征利用和ERC当前研究中的模式间冗余性方面的挑战.
主要方法:
- 使用跨模式封闭注意力机制提取和融合视觉,文字和听觉特征.
- 实施跨模式指导门机制,以加强单模式特征和改善双模式融合.
- 建议采用双重监督的跨模式蒸损耗函数,以减少冗余并增强特征歧视.
主要成果:
- 与IEMOCAP和MELD数据集的现有方法相比,提出的方法获得了更高的准确性.
- 该系统显示了可比的F1分数,表明有效捕获多式联运依赖.
- 实验结果验证了封闭注意力和蒸损失在平衡模式贡献中的有效性.
结论:
- 开发的跨模式封闭注意力机制显著提高了多模式情绪识别的准确性和稳定性.
- 这种新的方法有效地解决了在特征融合和冗余减少方面的关键挑战.
- 这项研究有助于通过增强对话情感理解来推进同情的人工智能系统的发展.
相关概念视频
Labeling Emotion
237
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
237
Emotional Expression
372
Emotional expression encompasses how individuals convey their emotions through verbal communication and non-verbal cues. These non-verbal actions include facial expressions, body language, and physical gestures, such as frowning or smiling. Among these, facial expressions play a crucial role in emotional expression and are understood universally, indicating a biological basis for how humans communicate emotions.
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
372
Physiology of Emotion
1.4K
The physiology of emotions is a multifaceted process involving the autonomic nervous system, brain structures, hormones, and neurotransmitters. This intricate interplay dictates how emotions manifest in the body and influence behavior.
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
1.4K
Coping Strategies: Emotion Focused
147
Emotion-focused coping refers to a set of strategies aimed at managing the emotional impact of stressors, rather than directly addressing their causes. This approach involves altering one's emotional response to stressful situations to reduce their psychological effects. For example, individuals might talk with a friend or engage in activities like journaling to express their feelings. Such actions can help achieve emotional clarity or release, providing the psychological stability needed...
147
Cognitive Theories: Schachter-Singer Theory of Emotion
589
Stanley Schachter and Jerome Singer proposed the two-factor theory of emotion, which emphasizes the interplay between physiological arousal and cognitive labeling in forming emotional experiences. This theory suggests that emotions are not simply a result of physiological responses but rather a combination of these responses and the individual's cognitive interpretation of them.
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
589
Facial Feedback Hypothesis
246
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
246

