一种多模型深度学习方法用于人类情绪识别和识别
Lalitha Arumugam1, Samydurai Arumugam2, Pabitha Chidambaram1
1Department of Computer Science and Engineering, SRM Valliammai Engineering College, Chennai, Tamil Nadu India.
Cognitive neurodynamics
|August 5, 2025
概括
这项研究引入了使用音频,视觉和文本数据进行情绪识别的新框架. 音频,视觉和文本情绪融合网络实现了高精度的实时情绪分析.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 情感计算是一种情感计算.
背景情况:
- 由于多模式表达 (语音,面部,文本) 导致情绪识别具有挑战性.
- 现有的方法很难有效地整合这些不同的情绪输入.
研究的目的:
- 引入一个新的框架,音频,视觉和文本情感融合网络 (AVT-EFN).
- 通过有效地结合多式联络输入来增强情绪分析.
主要方法:
- 图表注意力基于网络的变压器网络用于面部情绪分析.
- 混合 Wav2Vec 2.0 和卷积神经网络用于音频特征提取.
- 双向编码器表示来自变压器的双向门式反复单元,用于文本语义.
- 基于注意力的新机制,用于融合多式联运特征并改善跨式联运互动.
主要成果:
- 实现了98.7%的整体准确性.
- 证明了高精度 (98.2%),回忆 (97.2%) 和F1得分 (97.49%).
结论:
- 拟议的AVT-EFN框架对于多式联络情绪识别是有效和高效的.
- 基于注意力的融合机制增强了跨模式交互和情感背景理解.
更多相关视频
05:51Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
Published on: May 15, 2016
9.1K
05:41A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis
Published on: February 6, 2020
9.5K
相关概念视频
Labeling Emotion
237
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
237
Physiology of Emotion
1.4K
The physiology of emotions is a multifaceted process involving the autonomic nervous system, brain structures, hormones, and neurotransmitters. This intricate interplay dictates how emotions manifest in the body and influence behavior.
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
1.4K
Cognitive Theories: Schachter-Singer Theory of Emotion
591
Stanley Schachter and Jerome Singer proposed the two-factor theory of emotion, which emphasizes the interplay between physiological arousal and cognitive labeling in forming emotional experiences. This theory suggests that emotions are not simply a result of physiological responses but rather a combination of these responses and the individual's cognitive interpretation of them.
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
591
Empathy
9.7K
Some researchers suggest that altruism operates on empathy. Empathy is the capacity to understand another person’s perspective, to feel what he or she feels. An empathetic person makes an emotional connection with others and feels compelled to help (Batson, 1991). Empathy can be expressed in several ways, including cognitive, affective, and motor.
9.7K
Cognitive Theories: Lazarus Mediational Theory of Emotion
1.2K
Richard Lazarus' cognitive mediational theory highlights the pivotal role of cognitive appraisal in shaping emotional responses. According to this theory, the evaluation of a stimulus — based on personal values, goals, beliefs, and expectations — mediates the emotional response. This appraisal process is immediate and often occurs unconsciously, influencing the intensity and nature of the resulting emotion.
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
1.2K
Introduction to Motivation and Emotion
507
Motivation is a multifaceted process that drives behavior toward fulfilling various physiological or psychological needs. This process involves initiating, guiding, and maintaining specific actions influenced by internal and external factors. For example, when someone feels hungry while watching television, hunger is a motivator, prompting the individual to get up, walk to the kitchen, and find something to eat. In this instance, hunger initiates and sustains the behavior necessary to meet the...
507
