用大型语言模型评估韩国政治话语中的性别敏感性的基准数据集
Sunkyoung Park1, Eunbi Cho2, Chan Young Jung2
1Division of Global Korean Studies, Korea University, Seoul, Korea.
Scientific data
|December 11, 2025
概括
我们介绍KOGENT,这是一个数据集,用于评估韩国政治言论中的性别敏感性. 大型语言模型显示出希望,但与文化细微差别和背景作斗争.
科学领域:
- 计算语言学 计算语言学
- 政治科学 政治科学是指政治学.
- 性别研究是性别研究.
背景情况:
- 大型语言模型 (LLM) 越来越多地用于政治话语分析.
- 评估LLM在政治演讲中检测文化特异性性别敏感性的能力尚未得到充分研究.
研究的目的:
- 介绍KOGENT,这是一个新的基准数据集,用于分析韩国政治话语中的性别敏感性.
- 评估高级LLM (GPT-4.1,GPT-4o) 在此数据集中识别性别敏感性的表现.
主要方法:
- KOGENT包括1,222个韩国国民议会成绩单 (6,024个发言),涵盖1948年至2024年.
- 语句根据性别敏感性 (高/低) 和目标群体进行注释,并具有高跨编码协议.
- 在KOGENT数据集上使用零射击和少数射击学习方法评估了LLM.
主要成果:
- GPT-4.1的F1得分分别为87.5% (零射击) 和91.2% (18射击).
- 在GPT-4o中,F1得分为90.4% (零射击) 和91.1% (18射击).
- 这两种模型都在域内示例上表现得更好,但在文化细微差别和背景方面面临挑战.
结论:
- KOGENT作为一个强大的基准来评估韩国政治言论中的性别敏感性.
- 该研究强调了LLM需要改善社会文化对准,以便对政治话语进行细微分析.
- 需要进一步的研究来解决LLM在理解文化背景和复杂不平等方面的局限性.
相关概念视频
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Language and Cognition
688
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
688
Factorial Design
13.7K
Factorial Analysis is an experimental design that applies Analysis of Variance (ANOVA) statistical procedures to examine a change in a dependent variable due to more than one independent variable, also known as factors. Changes in worker productivity can be reasoned, for example, to be influenced by salary and other conditions, such as skill level. One way to test this hypothesis is by categorizing salary into three levels (low, moderate, and high) and skills sets into two levels (entry level...
13.7K
Stereotypes, Prejudice, and Discrimination
94.8K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
94.8K
Socioemotional Experience and Gender Development
250
Social-emotional experiences and cultural influences play significant roles in shaping gender development. During middle childhood, from ages 6 to 11, peer groups become dominant in reinforcing gender norms. Children in this age group often align with same-gender peer groups, which actively encourage behaviors that conform to traditional gender roles. For instance, boys may be discouraged from engaging in activities perceived as feminine, reinforcing culturally dictated norms about masculinity...
250
Language Development
804
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
804

