语音情感识别与轻量级深度神经合奏模型使用手工制作的功能
Jaher Hassan Chowdhury1, Sheela Ramanna2, Ketan Kotecha3
1The University of Winnipeg, 515 Portage Avenue, Winnipeg, Manitoba, Canada.
Scientific reports
|April 8, 2025
概括
这项研究表明,使用手工制作的功能进行语音情感识别 (SER) 的轻量级组合模型,优于基于光谱图的方法. 仔细的微调显著提高了情绪检测任务的性能.
科学领域:
- 人工智能的人工智能
- 人与计算机的交互
- 信号处理 信号处理
背景情况:
- 语音情感识别 (SER) 对医疗保健,HCI和机器人技术至关重要.
- 在SER的挑战包括数据稀缺和复杂的特征提取.
- 现有的方法通常依赖于像光谱图这样的自动特征提取.
研究的目的:
- 为了调查一个轻量级的深度神经元组合模型 (CNN & CNN_Bi-LSTM) 与手工制作的特征是否超过了SER的自动特征提取方法.
- 评估微调技术 (如学习速度调度器和规范化) 对模型性能的影响.
- 在多样化,公开可用的 SER 数据集上验证拟议的模型.
主要方法:
- 开发了一个CNN和CNN_Bi-LSTM组合模型.
- 使用了手工制作的音频功能:零交叉率 (ZCR),根平均平方误差 (RMSE),克罗玛短时间里埃变换 (STFT) 和梅尔频率塞普斯特尔系数 (MFCC).
- 在五个数据集 (RAVDESS,TESS,SAVEE,CREMA-D,EmoDB) 中使用学习率调度器,规范化和LIME技术进行模型解释.
主要成果:
- 拟议的整体模型始终优于单个模型和基于光谱的方法.
- 在关键指标上取得了卓越的表现:准确性,AUC-ROC,AUC-PRC和F1得分.
- LIME分析为模型的预测提供了可解释性.
结论:
- 轻量级的组装模型与精心挑选的手工制作特征对SER非常有效.
- 微调策略对于优化情绪识别中的深度学习模型至关重要.
- 提出的方法提供了一个强大的和可解释的解决方案,用于从语音中自动检测情绪.
相关概念视频
01:22Motional Emf
3.1K
Magnetic flux depends on three factors: the strength of the magnetic field, the area through which the field lines pass, and the field's orientation with respect to the surface area. If any of these quantities vary, a corresponding variation in magnetic flux occurs. If the area through which the magnetic field lines are passing changes, then the magnetic flux also changes. This change in the area can be of two types: the flux through the rectangular loop increases as it moves into the...
3.1K
01:14Muscles for Facial Expressions
1.7K
The craniofacial muscles are a collection of approximately 20 thin skeletal muscles situated beneath the skin of the face and scalp. These muscles, primarily responsible for the vast array of human facial expressions, originate from the bones or fibrous structures of the skull and extend outwards to connect with the skin. While most skeletal muscles in the body are enveloped in thick fascia, facial muscles generally have a more delicate fascial covering, with the buccinator muscle being a...
1.7K
01:24Facial Feedback Hypothesis
89
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
89
02:34Empathy
9.5K
Some researchers suggest that altruism operates on empathy. Empathy is the capacity to understand another person’s perspective, to feel what he or she feels. An empathetic person makes an emotional connection with others and feels compelled to help (Batson, 1991). Empathy can be expressed in several ways, including cognitive, affective, and motor.
9.5K