弥合文本和语音的情感理解:一个可解释的多式变压器融合框架与统一的音频文本归属
Ashutosh Pandey1, Jasmeet Singh1, Maninder Kaur1
1Computer Science Engineering Department, Thapar Institute of Engineering and Technology, Patiala 147001, Punjab, India.
这项研究介绍了一种可解释的情感识别人工智能模型,将语言和语音线索结合起来,以获得更好的准确性. 该框架使用多式变压器和可解释AI (XAI) 来理解情感沟通.
科学领域:
- 人工智能的人工智能
- 认知科学 认知科学
- 计算语言学 计算语言学
背景情况:
- 交谈互动为研究情感提供了丰富的语言和声音线索.
- 现有的情感识别模型往往专注于单一的模式,限制了对情感沟通的理解.
研究的目的:
- 开发一个可解释的多式联接变压器框架,用于高级情感理解.
- 整合文本语义和声学曲调,以进行全面的情感分析.
主要方法:
- 在文本语义上使用了RoBERTa,在声乐中使用了WavLM.
- 将两个模式投射到一个共享的潜在空间中,以便进行互补的分析.
- 嵌入式可解释的人工智能 (XAI) 技术,如集成梯度和封闭.
主要成果:
- 在五个情感类别中达到0.83准确度.
- 证明了模型能够捕捉语言和语音的互补贡献的能力.
- 成功地将预测归因于特定的语言标记和prosodic模式.
结论:
- 多式人工智能系统提高了情绪识别的准确性和透明度.
- 可解释的人工智能技术将计算机制与人类情感感知相协调.
- 这项工作支持以人为中心的情感识别,通过透明的人工智能.
更多相关视频
11:39Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
07:21Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy
Published on: June 27, 2025
相关概念视频
Attribution Theory
Transformers
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
Labeling Emotion
Cognitive Theories: Schachter-Singer Theory of Emotion
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Cognitive Theories: Lazarus Mediational Theory of Emotion
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
Theory of Attribution I: Correspondent Inference Theory
