在阿拉伯语中检测仇恨言论:库的设计,构建和评估
Ashraf Ahmad1, Mohammad Azzeh2, Eman Alnagi1
1Department of Computer Science, Princess Sumaya University for Technology (PSUT), Amman, Jordan.
Frontiers in artificial intelligence
|March 6, 2024
概括
研究人员创建了一个大型的阿拉伯仇恨言论数据集,以更好地在线检测. 这一数据集有助于机器学习模型在各种方言中识别有害内容,提高准确性.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 计算语言学 计算语言学
背景情况:
- 由于语言的多样性,阿拉伯语的仇恨言论检测是复杂的.
- 现有的方法缺乏全面的阿拉伯仇恨言论数据集.
- 在线平台需要有效的工具来打击有害内容.
研究的目的:
- 介绍一个新的,大规模的,多类的阿拉伯仇恨言论数据集.
- 为了评估阿拉伯语仇恨言论识别的机器学习模型性能.
- 为了促进阿拉伯语的进一步研究,在线内容调节.
主要方法:
- 开发了一个公开的数据集,包含403,688条注释的阿拉伯语推文.
- 使用的文本表示模型:Word2Vec,TF-IDF和AraBert.
- 评估了七个机器学习分类器:SVM,LR,NB,RF,AdaBoost,XGBoost和CatBoost. 这些分类器包括:
主要成果:
- 这种新型数据集在仇恨言论检测任务中被证明是有效的.
- 在具有挑战性的,非结构化的文本中取得了令人鼓舞的评估结果.
- 该数据集支持对各种机器学习模型进行可靠的评估.
结论:
- 创建的数据集是阿拉伯仇恨言论研究的宝贵资源.
- 机器学习模型展示了有效识别阿拉伯仇恨言论的潜力.
- 现在可以对这一关键领域进行进一步的学术研究.
更多相关视频
相关概念视频
Surveys
14.3K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.3K
Detection of Black Holes
1.7K
Although black holes were theoretically postulated in the 1920s, they remained outside the domain of observational astronomy until the 1970s.
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
1.7K
Types of Errors: Detection and Minimization
8.7K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
8.7K
Detection of Gross Error: The Q Test
7.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
7.1K
Classification of Signals
1.6K
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
1.6K
Language and Cognition
881
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
881


