在具有长期依赖性的认知任务上训练生物学上可信的循环神经网络
Wayne W M Soo1, Vishwa Goudar2, Xiao-Jing Wang2
1Department of Engineering, University of Cambridge.
bioRxiv : the preprint server for biology
|October 24, 2023
概括
研究人员开发了一种新方法来训练用于认知任务的循环神经网络 (RNN). 这种方法可以增强RNN的功能.
科学领域:
- 计算神经科学是一种神经科学.
- 认知科学 认知科学
- 机器学习 机器学习
背景情况:
- 循环神经网络 (RNN) 对于模拟认知中的神经假设是有价值的.
- 由于高效的算法和深度学习库,RNN的培训是可以访问的.
- 一个关键的挑战是训练RNN来完成长时间依赖的任务,而没有生物学上不可思议的门机制.
研究的目的:
- 开发一种生物学上可信的方法来培训RNN在具有长时间依赖性的认知任务上.
- 克服缺乏实验支持的标准RNN (例如,LSTM,GRU) 的局限性.
- 为了使RNN模型和生物神经电路之间的直接比较.
主要方法:
- 引入了专业的跳过连接通过时间,以促进任务相关的动态.
- 利用这些跳过连接,使RNN能够学习具有长期依赖性的任务.
- 在培训后恢复到原来的RNN架构以保持生物可信性.
主要成果:
- 这种新的方法使RNN能够成功地学习以前难以解决的复杂的认知任务.
- 实现了减少训练步骤和缩短计算时间,特别是在需要长时间整合或记忆的任务中.
- 与传统方法相比,在具有长期依赖性的任务中表现得更好.
结论:
- 拟议的方法扩大了生物学上可信的RNN模型的功能.
- 促进了关于神经机制的理论的发展,这些神经机制是长期依赖的计算的基础.
- 支持创建大脑更现实的计算模型.
相关概念视频
Long-term Potentiation
55.3K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.3K
Long-term Depression
2.5K
Long-term depression, or LTD, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTD is the process of synaptic weakening that occurs over time between pre and postsynaptic neuronal connections. The synaptic weakening of LTD works in opposition to synaptic strengthening by long-term potentiation (LTP) and together are the main mechanisms that underlie learning and memory.
Calcium Ion Concentration Mechanism
If over...
Calcium Ion Concentration Mechanism
If over...
2.5K
Long-Term Memory
174
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
174
Cognitive Learning
249
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
249
Storage
87
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
87
Higher Mental Functions of Brain: Learning and Memory
837
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
837


