在阅读的认知模型中,语言模型的表现优于cloze可预测性
Adrielli Tina Lopes Rego1, Joshua Snell2, Martijn Meeter1
1Department of Education, Vrije Universiteit Amsterdam, and LEARN! Research Institute, Amsterdam, The Netherlands.
PLoS computational biology
|September 25, 2024
概括
大型语言模型 (LLM) 在阅读中比传统的闭门任务更好地衡量单词的可预测性. 这提高了阅读和眼动控制的计算模型.
科学领域:
- 认知科学 认知科学
- 计算语言学 计算语言学
- 心理语言学 心理语言学
背景情况:
- 词的可预测性对于阅读至关重要,但目前的理论缺乏复杂的解释.
- 计算阅读模型经常使用cloze规范,但它们的确切含义尚不清楚.
研究的目的:
- 调查大型语言模型 (LLM) 是否可以为阅读模型提供更准确的词语可预测性测量.
- 探索字的可预测性和眼睛在阅读过程中的运动之间的相互作用.
主要方法:
- 在LLMs中实施了用于上下文预测的新型并行分级机制.
- 使用OB1阅读器,阅读的认知模型,用于模拟.
- 与眼动数据相匹配的比较模型使用cloze任务与LLM (GPT-2,LLaMA) 的可预测性值.
主要成果:
- 从LLM获得的可预测性值与人类眼动数据相比,更适合于cloze规范.
- 使用LLM可预测性时,根平均平方误差较低.
结论:
- 通过结合高阶语言处理,LLM可以增强认知阅读模型.
- 在阅读的计算模型中,LLM可预测性为单词可预测性提供了更精确的代理.
相关概念视频
Language and Cognition
336
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
336
Cognitive Learning
229
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
229
Typical Model Studies
344
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.
344
Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
Associative Learning
309
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
309
Hindsight Biases
3.4K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.4K


