在人工和生物神经网络中,语言语境的逐渐积累
Refael Tikochinski1, Ariel Goldstein2, Yoav Meiri3
1The Faculty of Data and Decisions Sciences, Technion - Israel Institute of Technology, Haifa, Israel. Rafyona@gmail.com.
Nature communications
|January 17, 2025
概括
人类大脑逐步整合上下文,与使用并行处理的大型语言模型 (LLM) 不同. 模仿这种增量方法的新LLM模型更好地预测叙事理解期间的神经活动.
科学领域:
- 认知神经科学 认知神经科学
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
背景情况:
- 大型语言模型 (LLM) 擅长预测用于叙事处理的神经信号.
- 在LLM的平行上下文整合不同于大脑的时间处理.
研究的目的:
- 研究大脑如何逐步整合短期和长期的环境.
- 开发和评估基于LLM的增量上下文模型,用于预测神经活动.
主要方法:
- 利用了来自219名参与者听取口语叙述的fMRI数据.
- 评估了不同背景窗口大小的LLM预测准确性.
- 介绍并测试了一种增量上下文法学士,将短期输入与动态的先前上下文摘要结合起来.
主要成果:
- 只有在短的上下文窗口 (几十个单词) 中,LLM才能准确地预测神经活动.
- 新的增量上下文LLM显著改善了神经活动的预测.
- 在对长时间处理至关重要的高阶大脑区域观察到增强的预测.
结论:
- 大脑在时间信息整合方面采用了层次化的增量机制.
- 这种机制可以在较长时间内灵活地处理上下文.
- 研究结果为认知神经科学提供了洞察力,并推动了用于自然语言理解的AI开发.
更多相关视频
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
494
相关概念视频
Neuroplasticity
281
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
281
Neural Circuits
1.0K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.0K
Long-term Potentiation
54.7K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
54.7K
Language Development
314
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
314
Language and Cognition
321
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
321
Integration of Synaptic Events
1.4K
Synaptic integration mainly includes the summation of graded potentials. Graded potentials, regardless of their type, cause subtle alterations in membrane voltage, resulting in either depolarization or hyperpolarization. These incremental changes, when combined or summed, can propel the neuron toward its threshold. Consider, for example, a membrane experiencing a +15 mV shift, causing it to depolarize from -70 mV to -55 mV. In this scenario, graded potentials govern the membrane's ability...
1.4K
