"LLM词语世界"由大型语言模型产生的英语自由协会规范
Katherine Abramski1, Riccardo Improta2, Giulio Rossetti3
1University of Pisa, Department of Computer Science, Pisa, Italy. katherine.abramski@phd.unipi.it.
Scientific data
|May 16, 2025
概括
研究人员创建了"LLM World of Words" (LWOW) 数据集,从大型语言模型 (LLM) 中生成自由关联. 这一数据集可以研究LLM知识和偏见,与人类语义网络相提并论.
科学领域:
- 认知科学 认知科学
- 计算语言学 计算语言学
- 人工智能的人工智能
背景情况:
- 自由协会任务是心理学和语言学中探索人类概念知识组织的既定方法.
- 在大型语言模型 (LLM) 中调查偏见是人工智能研究的一个不断增长的领域.
- 由LLM产生的缺乏大规模,人类可比的自由协会规范,阻碍了对LLM知识和偏见的研究.
研究的目的:
- 为满足LLM产生的自由协会规范的需求.
- 为研究LLM知识和偏见创建与人类生成的规范可比的数据集.
- 为了实现基于网络的分析语义记忆在人类和LLMs.
主要方法:
- 开发了"LLM词语世界" (LWOW) 数据集,该数据集以人类制造的"小词语世界" (SWOW) 规范为模型.
- 提醒了三个LLM (Mistral,Llama3,Haiku) 与来自SWOW数据集的近12,000个提示词.
- 从生成的关联数据中构建语义内存的网络模型.
- 通过在网络模型中模拟语义原始化效应来验证LWOW数据集.
主要成果:
- 成功生成了三个新的,大规模的LLM生成的自由关联规范数据集 (LWOW).
- 在LLMs和人类中构建了代表语义知识的可比网络模型.
- 通过语义启动模拟验证了LWOW数据集的实用性.
结论:
- LWOW数据集为人类和LLMs之间的语义知识的比较研究提供了宝贵的资源.
- 这些数据集有助于调查LLM中编码的隐性偏见.
- 该方法为创建和分析LLM生成的语义规范提供了一个框架.
相关概念视频
Language and Cognition
301
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
301
Associative Learning
270
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
270
Higher Mental Functions of the Brain: Language
700
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
700
Genetic Lingo
98.8K
Overview
98.8K
Language Development
292
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
292
Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K


