使用DistilBERT和ALBERT优化知识蒸,以有效地识别社交媒体情绪.
Muhammad Hussain1, Caikou Chen2, Muzammil Hussain3
1College of Information and Artificial Intelligence, Yangzhou University, Yangzhou, 225000, People's Republic of China.
我们使用知识蒸开发了一种高效的情感识别方法. 这种方法可以显著降低模型大小和延迟,同时保持高精度,非常适合实时应用.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 情感计算是一种情感计算.
背景情况:
- 在社交媒体文本中准确识别情绪对于各种应用至关重要,但面临着计算复杂性和阶级不平衡等挑战.
- 变压器模型提供高性能,但对于实时,资源有限的环境来说太大,太慢.
研究的目的:
- 为高效的情感识别提出一种新的知识蒸框架.
- 从一个大型BERT基模型转移知识到较小的DistilBERT和ALBERT模型.
主要方法:
- 实施了知识蒸框架,具有混合损失函数 (焦点损失和KL分歧),以改善少数阶级的认可.
- 利用注意力-头部对齐来实现有效的知识传输和语义维护数据增强,以解决阶级不平衡.
- 在两个大规模的社交媒体情绪数据集 (Twitter情绪和社交媒体情绪) 上训练和评估模型.
主要成果:
- 蒸模型实现了接近教师的表现,精度下降最小 (<1%和<6%).
- 模型大小减少了40%,推断延迟减少了3.2×.
- 显著改善了少数群体情绪类的F1分数.
结论:
- 拟议的知识蒸框架能够有效和准确地识别情绪.
- 这种方法克服了大型变压器模型的局限性,促进了边缘计算和移动应用中的部署.
- 在社交媒体中高效地识别情绪的新技术. 文本.
更多相关视频
05:51Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
Published on: May 15, 2016
07:12Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation
Published on: August 26, 2016
相关概念视频
Labeling Emotion
Cognitive Theories: Schachter-Singer Theory of Emotion
Physiological Arousal and Cognitive Labeling
According to this theory, when an individual experiences...
Facial Feedback Hypothesis
Physiology of Emotion
Autonomic Nervous System
The autonomic nervous system (ANS) plays a critical role in emotional responses by regulating involuntary physiological functions. It consists of two main components: the sympathetic and parasympathetic systems. The sympathetic system...
Nonconscious Mimicry
Cognitive Theories: Lazarus Mediational Theory of Emotion
Cognitive Appraisal and Emotional Response
Lazarus proposed that...
