神经回路中的语言的超预测学习模型
Chan Li1,2, Junbin Qiu1, Haiping Huang1,3
1PMI Laboratory, School of Physics, Sun Yat-sen University, Guangzhou 510275, People's Republic of China.
Physical review. E
|May 17, 2024
概括
这项研究引入了一种新的超预测学习模型,其灵感来源于大脑的预测编码框架. 该模型成功处理了序列数据,为大脑计算和人工智能之间的联系提供了洞察力,特别是大型语言模型.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 大型语言模型 (LLM) 使用自我注意机制展示了显著的能力.
- 人类大脑的语言处理可能与LLM原则不同,引发了关于大脑计算和人工自我监督的辩论.
- 预测编码框架是一个突出的大脑计算假设,但它在语言处理和信用分配中的作用尚不清楚.
研究的目的:
- 在预测编码框架内提出和验证一个平均场学习模型.
- 调查大脑计算,下一个令牌预测和一般智能之间的联系.
- 探索预测编码和信用分配在语言处理中的作用.
主要方法:
- 开发了一个基于预测编码框架的平均场学习模型.
- 假设突触重量遵循尖峰和板块分布,只训练分布.
- 验证了手写数字分类 (顺序像素输入) 和语言体 (玩具和真实) 的元预测学习模型.
主要成果:
- 该模型成功地对连续数据进行了分类,包括手写数字和语言体.
- 在学习后,大多数连接变得决定性,而输出连接显示出更高的可变性.
- 网络组合性能随着数据负载的不断扩展,随着更多的培训数据的改善,反映了LLM的新兴行为.
结论:
- 提出的超预测性学习模型为理解语言处理中的大脑计算提供了一个框架.
- 这些发现表明,预测编码,LLM中的下一个令牌预测和新兴的一般智能之间存在潜在的联系.
- 该模型为未来的研究提供了基础,将神经科学与人工智能联系起来.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.9K
10:42A Lateralized Odor Learning Model in Neonatal Rats for Dissecting Neural Circuitry Underpinning Memory Formation
Published on: August 18, 2014
8.9K
相关概念视频
Higher Mental Functions of the Brain: Language
796
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
796
Language and Cognition
343
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
343
Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
Long-term Potentiation
55.1K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.1K
Lateralization
327
Brain lateralization refers to the division of mental processes and functions between the two hemispheres of the brain, a phenomenon that optimizes neural efficiency and underpins complex abilities in humans. This specialization allows each hemisphere to perform tasks where it has a comparative advantage, facilitating more refined cognitive capabilities across different domains.
327
Neural Regulation
39.4K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.4K
