一个双语马来语-英语社交媒体数据集用于二元仇恨言论检测
Jun-Chen Tan1, Lee-Yeng Ong1, Meng-Chew Leow1
1Faculty of Information Science and Technology, Multimedia University, Melaka, 75450 Malaysia.
Data in brief
|October 29, 2025
概括
这项研究引入了一个新的双语马来语-英语数据集,用于检测在线仇恨言论,这对于改善多语言社交媒体环境中的安全至关重要. 该资源支持用于代表性不足的语言的机器学习.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 计算语言学 计算语言学
- 社交媒体分析 社交媒体分析
背景情况:
- 在线仇恨言论对用户安全和社会凝聚力构成重大威胁.
- 现有的仇恨言论数据集通常是单语言的,忽略了像马来语这样的代表性不足的语言.
- 东南亚语言缺乏足够的NLP研究资源,阻碍了仇恨言论的检测.
研究的目的:
- 通过创建一个平衡的,质量控制的资源来解决多语言仇恨言论数据集的差距.
- 支持机器学习应用程序,用于在低资源,多语言环境中检测仇恨言论.
- 为实用,早期的仇恨言论检测系统提供二进制分类任务.
主要方法:
- 策划了来自五个公共来源的26,985个双语马来语-英语社交媒体文本.
- 结合人类注释与受控伪标签,用于高可靠性数据.
- 通过严格的过和验证流程,确保数据质量.
主要成果:
- 开发了一个均衡的数据集,包含13,609个英语和13,376个马来语文本.
- 提供 UTF-8 编码的 CSV 格式的数据,带有明确的标签和元数据.
- 数据集有助于培训多语言模型和对跨语言NLP进行基准测试.
结论:
- 该数据集满足了在英语和马来语中检测仇恨言论的实际需求.
- 它使得有效的多语言仇恨言论检测系统的开发成为可能.
- 本资源支持针对英语和马来语社区的教育NLP倡议.
相关概念视频
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Group Polarization
38.4K
Group polarization is the strengthening of an original group attitude following the discussion of views within a group (Teger & Pruitt, 1967). That is, if a group initially favors a viewpoint, after discussion the group consensus is likely a stronger endorsement of the viewpoint. Conversely, if the group was initially opposed to a viewpoint, group discussion would likely lead to stronger opposition.
38.4K
Attitudes
32.7K
Attitude is our evaluation of a person, an idea, or an object. We have attitudes for many things ranging from products that we might pick up in the supermarket to people around the world to political policies. Typically, attitudes are favorable or unfavorable: positive or negative (Eagly & Chaiken, 1993). And, they have three components: an affective component (feelings), a behavioral component (the effect of the attitude on behavior), and a cognitive component (belief and knowledge;...
32.7K
Stereotypes, Prejudice, and Discrimination
94.9K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
94.9K
Bias
7.2K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
7.2K
Detection of Black Holes
2.5K
Although black holes were theoretically postulated in the 1920s, they remained outside the domain of observational astronomy until the 1970s.
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
2.5K
