基于面部表情,语音和EEG的多式情绪识别
Jiahui Pan1, Weijie Fang1, Zhihang Zhang1
1School of SoftwareSouth China Normal University Guangzhou 510631 China.
IEEE open journal of engineering in medicine and biology
|June 20, 2024
概括
深度情感是一种新的多式联动情感识别 (MER) 系统,有效地整合了面部表情,语音和脑电图 (EEG) 数据,以提高准确性. 这种深度学习方法提供了强大的实时情绪检测,推进了人机交互.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 情感识别对于人机交互至关重要.
- 现有的方法在多模式集成和实时处理方面面临挑战.
- 深度学习模型需要显著的计算能力,影响强度.
研究的目的:
- 提出Deep-Emotion,一个基于深度学习的多式联络式情绪识别 (MER) 系统.
- 从面部表情,言语和EEG中自适应地整合歧视特征.
- 为了提高表现,实时检测和情绪识别的稳定性.
主要方法:
- 开发了一个三分支框架:改善了面部表情的GhostNet,用于语音的轻量级完全卷积神经网络 (LFCNN),以及用于EEG的树状LSTM (tLSTM).
- 采用决策层面的融合来整合三个模式的结果.
- 利用改进的GhostNet来缓解过度拟合和提高分类准确性.
主要成果:
- 在CK+,EMO-DB和MAHNOB-HCI数据集上的广泛实验验证实了深度情绪方法.
- 证明了提出的MER方法的先进性质和优越性.
- 通过多模式融合实现了全面而准确的情感识别.
结论:
- 深度情感方法是先进的,有效的多模式情感识别.
- 提出的方法是可行的,并且优于现有的方法.
- 深度情感通过准确而强大的情感检测来增强人机交互.
相关概念视频
Facial Feedback Hypothesis
139
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
139
Emotional Expression
202
Emotional expression encompasses how individuals convey their emotions through verbal communication and non-verbal cues. These non-verbal actions include facial expressions, body language, and physical gestures, such as frowning or smiling. Among these, facial expressions play a crucial role in emotional expression and are understood universally, indicating a biological basis for how humans communicate emotions.
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
202


