大型语言模型可以帮助增强英语心理语言数据集吗?
1Department of Cognitive Science, UC San Diego, 9500 Gilman Dr., La Jolla, CA, 92093-0515, USA. sttrott@ucsd.edu.
Behavior research methods
|January 23, 2024
概括
像GPT-4这样的大型语言模型 (LLM) 可以产生可靠的心理语言规范,匹配人类的协议水平,用于诸如单词相似性和感觉运动关联等任务. 这为语言和认知研究提供了一个可扩展的替代性,而不是昂贵的人类数据收集.
科学领域:
- 认知科学 认知科学
- 计算语言学 计算语言学
- 心理语言学 心理语言学
背景情况:
- 心理语言学数据集或规范对于语言和认知研究至关重要.
- 收集这些规范的人类判断是耗时和昂贵的,特别是对于大规模或多维数据.
- 大型语言模型 (LLM) 是克服这些数据收集挑战的潜在解决方案.
研究的目的:
- 调查使用LLM,特别是GPT-4来生成英语的心理语言规范的可行性.
- 将LLM生成的语义判断与人类生成的数据 (黄金标准) 进行比较.
- 评估LLM生成规范在统计建模中的实用性,并确定潜在的局限性.
主要方法:
- GPT-4被用来收集英语单词的各种语义判断 (例如,单词相似性,上下文感觉运动关联,标志性).
- 由LLM产生的判断被系统地与既定的人类产生的规范进行了比较.
- 进行了替代分析,在统计模型中用LLM规范取代人类规范.
主要成果:
- 在不同数据集中,GPT-4的判断与人类的判断有积极的相关性.
- 在某些情况下,LLM协议水平达到或超过了平均人类互注释器协议.
- 替代分析表明,LLM规范通常保留统计模型中参数估计的方向,尽管大小可能会改变.
结论:
- 实际上,LLM可以有效地增强大规模心理语言数据集的创建,为人类数据收集提供一个经济有效和高效的替代方案.
- 在LLM和人类规范之间存在系统的差异,需要仔细考虑数据污染,LLM选择和有效性等因素.
- 该研究提供了有价值的LLM生成的规范数据,并讨论了它们未来在研究中使用的关键考虑.
更多相关视频
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
457
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.2K
相关概念视频
01:27Language and Cognition
346
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
346
01:22Language Development
367
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
367