代码混合揭晓:使用机器学习模型提高了在阿拉伯方言推特中检测仇恨言论的性能
Ali Alhazmi1,2, Rohana Mahmud1, Norisma Idris1
1Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur, Malaysia.
PloS one
|July 17, 2024
概括
这项研究增强了对阿拉伯社交媒体的仇恨言论检测,特别是通过代码混合. TF-IDF功能和SGD模型实现了98.21%的准确性,超过了以前的方法.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 计算语言学 计算语言学
背景情况:
- 社交媒体促进了国际沟通,但与仇恨言论作斗争.
- 由于方言和代码混合,阿拉伯语对仇恨言论的检测提出了独特的挑战.
- 现有的以英语为中心的方法对于细微的阿拉伯语内容是不够的.
研究的目的:
- 为了评估阿拉伯语仇恨言论检测的机器学习模型.
- 评估变异特征对检测代码混合仇恨言论的影响.
- 为了比较阿拉伯语和代码混合的仇恨言论数据集上的模型性能.
主要方法:
- 数据收集和预处理阿拉伯社交媒体文本.
- 特性提取,包括TF-IDF.
- 开发和评估各种机器学习分类模型.
- 与现有的仇恨言论检测研究进行比较.
主要成果:
- TF-IDF功能与SGD模型相结合,实现了最高的准确率98.21%.
- 与之前的三项研究相比,提出的方法显示出更高的性能.
- 在阿拉伯语推特中有效识别仇恨言论,包括代码混合实例.
结论:
- 具有适当功能的机器学习模型可以有效地检测阿拉伯的仇恨言论.
- TF-IDF和SGD模型组合为这个挑战提供了一个强大的解决方案.
- 这项研究为在多语言环境中自动检测仇恨言论提供了基础.
相关概念视频
01:27Air-entraining Agents
Air-entraining agents improve the durability and workability of concrete in climates with frequent freezing and thawing. These agents prevent cracks by introducing small air bubbles into the mix, creating spaces accommodating water expansion when temperatures drop. The air-entraining agents lower the surface tension of water, forming stable, small air bubbles. This method is more effective than having accidental large voids, as the intentional, smaller, and evenly distributed air voids improve...
01:27Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
01:14Impression Management Techniques IV: Altercasting
Altercasting is a strategic communication technique in which an individual imposes a specific identity or social role onto another person to influence their behavior and shape the interaction. By presuming a role—such as “responsible leader” or “patient person”—altercasting encourages the target to conform to that identity, often aligning their behavior with the expectations associated with the role. The power of this tactic lies in its subtlety; once a role is assigned, it becomes socially...