具有竞争力的可塑性,以降低学习的能量成本
Mark C W van Rossum1,2, Aaron Pache2
1School of Psychology, University of Nottingham, Nottingham, United Kingdom.
PLoS computational biology
|October 28, 2024
概括
大脑是大脑的大脑.
科学领域:
- 计算神经科学是一种计算神经科学.
- 人工智能的人工智能是人工智能.
- 代谢能源支出代谢能源支出
背景情况:
- 大脑功能受到计算和记忆形成所需能量的限制.
- 即使是简单的学习任务也会由于突触更新而产生显著的代谢成本.
- 高精度的学习,比如MNIST到95%,需要大量的突触修改.
研究的目的:
- 在feedforward神经网络中研究学习的能量需求.
- 开发用于突触可塑性的节能算法.
- 为了弥合人工和生物学习机制之间的理解.
主要方法:
- 开发了一种用于神经网络学习的节能源模型.
- 提出了两个可塑性限制算法: 1) 仅修改大型突触更新, 2) 将可塑性限制在必要的网络路径上.
- 将算法性能与标准反向传播进行比较.
主要成果:
- 塑性限制显著降低了学习期间的代谢能量消耗.
- 算法在学习时间上只增加了很少的时间,从而实现了相当大的节能.
- 生物网络往往比必要的更大,可以从这种可塑性约束中受益.
结论:
- 限制突触可塑性是一种有效的策略,可以节省学习中的代谢能量.
- 这些发现提高了对生物可塑性的理解,并改善了人工学习模型.
- 拟议的算法为节能硬件提供了潜在的好处.
相关概念视频
Neuroplasticity
303
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
303
Long-term Potentiation
2.7K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.7K
Cognitive Learning
222
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
222
Purposive Learning
101
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
101
Plasticity
2.1K
Plasticity is the property where an object loses its elasticity and undergoes irreversible deformation, even after the deformation forces are eliminated. If a material deforms irreversibly without increasing stress or load, then this is called ideal plasticity. For example, when a force is applied to an aluminum rod, it changes its shape, but it does not return to its original shape once the force is removed. Plastic deformation or ductility is thus a permanent deformation or change in the...
2.1K
Plastic Behavior
189
A material's elastic behavior is characterized by the disappearance of stress once the load is removed, allowing the material to return to its original state. However, when stress surpasses the yield point, yielding commences, marking the onset of plastic deformation or permanent set. This change from elastic to plastic behavior is influenced by the peak stress value and the duration before the load is removed. An intriguing observation occurs when a specimen is loaded, unloaded, and...
189


