具有切换单元的生物灵感架构可以学习跨背景的概括
Doris Voina1, Eric Shea-Brown2, Stefan Mihalas2
1Department of Applied Mathematics, Computational Neuroscience Center, University of Washington, Seattle, WA 98195, USA.
概括
这项研究介绍了一种生物灵感的瓶切换网络,以防止人工系统的灾难性遗忘. 新型架构使人工智能能够在不失去先前知识的情况下学习新语境,从而提高了概括能力.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 生物系统表现出了显著的上下文概括,这是当前人工系统缺乏的能力.
- 深度学习模型经常在遇到新的数据环境时遭受灾难性遗忘.
- 人工智能的持续学习旨在使模型能够连续学习,而不会对之前的任务造成性能下降.
研究的目的:
- 开发一种生物灵感的人工神经网络架构,克服灾难性遗忘.
- 增强人工系统的上下文概括能力.
- 研究特定网络模式在实现灵活学习中的作用.
主要方法:
- 在两个不同的背景 (噪音和CIFAR-10) 上使用具有不同透明度的MNIST数字来创建两个上下文.
- 训练浅层和深层网络,观察在上下文切换时的性能下降 (灾难性遗忘).
- 提出并实施了一个瓶切换网络 (切换网络),使用生物启发的"切换"单元.
主要成果:
- 交换网络成功地学习了新的背景,而不忘记以前的背景,减轻了灾难性的遗忘.
- 少数切换单元足以抑制冗余的背景功能,并使新学习成为可能.
- 瓶切换网络展示了与以前学习的类似的新背景的概括.
结论:
- 生物启发的网络图案,特别是具有"切换"单元的网络图案,可以有效地解决人工智能的灾难性遗忘问题.
- 对交换单元的反复连接增强了上下文概括,反映了生物视觉皮层网络.
- 拟议的交换网络为开发更灵活,更适应的人工智能系统提供了一个有希望的方向.
相关概念视频
Multi-input and Multi-variable systems
111
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
111
Neural Circuits
1.3K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.3K
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.1K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.1K
Generalization, Discrimination, and Extinction
583
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
583
Sequence Networks of Rotating Machines
107
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
107


