人間とマルチモダルのLLM間のビデオクリップから引き出された高次元感情構造の対応
Haruka Asanuma1, Naoko Koide-Majima2,3, Ken Nakamura4
1Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, 153-8902, Japan.
Scientific reports
|September 1, 2025
まとめ
マルチモダル大言語モデル (MLLMs) は,粗いカテゴリーレベルで人間の感情構造に強い類似性を示しています. しかし 単一項目レベルで 複雑さを捉えるのは困難です
科学分野:
- 認知科学
- 人工知能
- 感情的コンピューティング
背景:
- 人間の感情は 複雑で高度な構造を持ち 従来のモデルは 単純すぎます
- 感情の複雑さを理解することは 人間とコンピュータのやり取りや 心理学の研究を進める上で 極めて重要です
- 複合的な大きな言語モデル (MLLMs) の出現は,複雑な感情状態をモデル化するための能力を探求する機会を提供します.
研究 の 目的:
- 最先端のマルチモダル大型言語モデル (MLLMs) が人間の感情の高次元構造をどの程度捉えているかを調査する.
- MLLMによって生成された感情の見積もりと,様々なビデオ刺激における人間の自己報告された感情を比較する.
- 個々のビデオとビデオ間の関係レベルの両方でMLLMのパフォーマンスを評価し,粗いカテゴリーと単一の項目の精度を区別する.
主な方法:
- ビデオ刺激に対して,参加者が報告した感情評価とMLLMによって生成された感情評価 (例えば,Gemini,GPT) の比較.
- 個別ビデオレベルでの相関分析を使用して,感情構造の類似性の評価.
- Gromov-Wasserstein 最適トランスポートの応用 感情推論における単項と粗いカテゴリーレベルのパフォーマンスを区別する.
主要な成果:
- 人間とMLLMが推論した感情の構造は,相関レベルにおいて全体的に強い類似性が観察された.
- シングルアイテムの感情予測の精度は限られていましたが,MLLMはビデオのカテゴリー間の感情的な類似性を捉える上で実質的なパフォーマンスを示しました.
- これは,MLLMは,感情的な風景の広範な理解を反映した,粗いカテゴリーレベルで人間の感情的な経験を推論することができることを示しています.
結論:
- 現在の最先端のMLLMは 人間の感情の複雑で高次元的な構造を 粗略なカテゴリーレベルで捉えています
- MLLMは 単一の項目レベルで 完全な感情構造を正確に捉えるのに 限界があります
- これらの発見は,人間の感情状態の理解とモデル化におけるMLLMの潜在的および現在の限界を強調しています.
さらに関連する動画
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
681
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
関連する概念動画
Labeling Emotion
237
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
237
Cognitive Theories: Lazarus Mediational Theory of Emotion
1.2K
Richard Lazarus' cognitive mediational theory highlights the pivotal role of cognitive appraisal in shaping emotional responses. According to this theory, the evaluation of a stimulus — based on personal values, goals, beliefs, and expectations — mediates the emotional response. This appraisal process is immediate and often occurs unconsciously, influencing the intensity and nature of the resulting emotion.
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
1.2K
Physiology of Emotion
1.4K
The physiology of emotions is a multifaceted process involving the autonomic nervous system, brain structures, hormones, and neurotransmitters. This intricate interplay dictates how emotions manifest in the body and influence behavior.
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
1.4K
Empathy
9.7K
Some researchers suggest that altruism operates on empathy. Empathy is the capacity to understand another person’s perspective, to feel what he or she feels. An empathetic person makes an emotional connection with others and feels compelled to help (Batson, 1991). Empathy can be expressed in several ways, including cognitive, affective, and motor.
9.7K
Emotional Expression
367
Emotional expression encompasses how individuals convey their emotions through verbal communication and non-verbal cues. These non-verbal actions include facial expressions, body language, and physical gestures, such as frowning or smiling. Among these, facial expressions play a crucial role in emotional expression and are understood universally, indicating a biological basis for how humans communicate emotions.
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
Universal Facial Expressions
Psychologist Paul Ekman identified seven basic...
367
Cognitive Theories: Schachter-Singer Theory of Emotion
579
Stanley Schachter and Jerome Singer proposed the two-factor theory of emotion, which emphasizes the interplay between physiological arousal and cognitive labeling in forming emotional experiences. This theory suggests that emotions are not simply a result of physiological responses but rather a combination of these responses and the individual's cognitive interpretation of them.
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
579
