人类与多式LLM之间的视频剪辑引起的高维情感结构的对应性
Haruka Asanuma1, Naoko Koide-Majima2,3, Ken Nakamura4
1Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, 153-8902, Japan.
Scientific reports
|September 1, 2025
概括
多模式大语言模型 (MLLMs) 在粗略的类别层面上与人类情感结构有很强的相似性. 然而,他们很难在单个项目层面上捕捉完整的复杂性.
科学领域:
- 认知科学
- 人工智能
- 情感计算
背景情况:
- 人类的情绪具有复杂,高维的结构,
- 了解情感复杂性的细微差别对于推进人机交互和心理学研究至关重要.
- 多模态大语言模型 (MLLM) 的出现为探索它们模拟复杂情绪状态的能力提供了机会.
研究的目的:
- 研究最先进的多模式大语言模型 (MLLMs) 如何捕捉人类情绪的高维结构.
- 将MLLM产生的情绪估计与各种视频刺激中人类自我报告的情绪进行比较.
- 在单个视频和视频之间的关系层面评估MLLM性能,区分粗略类别和单项准确性.
主要方法:
- 参与者报告的情绪评级与MLLM产生的情绪估计 (例如Gemini,GPT) 的视频刺激的比较.
- 使用个人视频层面的相关性分析评估情感结构相似性.
- 应用Gromov-Wasserstein最佳传输来区分情感推断中的单项和粗略类别水平性能.
主要成果:
- 在相关水平上观察到人类和MLLM推断的情感结构之间存在强烈的整体相似性.
- 虽然单项情感预测的准确性有限,但MLLM在捕捉不同类别视频的情感相似性方面表现出很好的表现.
- 这表明MLLM可以在粗略的类别层面推断人类的情感体验,反映出对情感景观的广泛理解.
结论:
- 目前最先进的MLLM在粗略的层面上捕捉到人类情绪的复杂,高维度结构.
- 在严格的单项层面上,MLLM在准确地捕捉整个情感结构方面存在局限性.
- 这些发现突显了MLLM在理解和建模人类情感状态方面的潜在和当前的局限性.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
相关概念视频
Labeling Emotion
237
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
237
Cognitive Theories: Lazarus Mediational Theory of Emotion
1.2K
Richard Lazarus' cognitive mediational theory highlights the pivotal role of cognitive appraisal in shaping emotional responses. According to this theory, the evaluation of a stimulus — based on personal values, goals, beliefs, and expectations — mediates the emotional response. This appraisal process is immediate and often occurs unconsciously, influencing the intensity and nature of the resulting emotion.
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
1.2K
Physiology of Emotion
1.4K
The physiology of emotions is a multifaceted process involving the autonomic nervous system, brain structures, hormones, and neurotransmitters. This intricate interplay dictates how emotions manifest in the body and influence behavior.
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
1.4K
Empathy
9.7K
Some researchers suggest that altruism operates on empathy. Empathy is the capacity to understand another person’s perspective, to feel what he or she feels. An empathetic person makes an emotional connection with others and feels compelled to help (Batson, 1991). Empathy can be expressed in several ways, including cognitive, affective, and motor.
9.7K
Emotional Expression
367
Emotional expression encompasses how individuals convey their emotions through verbal communication and non-verbal cues. These non-verbal actions include facial expressions, body language, and physical gestures, such as frowning or smiling. Among these, facial expressions play a crucial role in emotional expression and are understood universally, indicating a biological basis for how humans communicate emotions.
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
367
Cognitive Theories: Schachter-Singer Theory of Emotion
579
Stanley Schachter and Jerome Singer proposed the two-factor theory of emotion, which emphasizes the interplay between physiological arousal and cognitive labeling in forming emotional experiences. This theory suggests that emotions are not simply a result of physiological responses but rather a combination of these responses and the individual's cognitive interpretation of them.
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
579
