在尖端网络中使用memristors进行持续学习的概率性元可塑性
Fatima Tuz Zohora1, Vedant Karia2, Nicholas Soures2
1Neuromorphic Artificial Intelligence Lab, University of Texas at San Antonio, San Antonio, TX, 78249, USA. fatimatuz.zohora@my.utsa.edu.
Scientific reports
|November 28, 2024
概括
这项研究介绍了边缘设备上的持续学习的概率性元可塑性. 这种新的方法可以实现高效的学习,而不会造成灾难性的遗忘,使用低精度的memristor重量,并减少记忆和能源消耗.
科学领域:
- 人工智能的人工智能
- 神经科学是一个神经科学.
- 材料科学 材料科学 材料科学
背景情况:
- 边缘设备需要不断学习能力,以适应动态环境.
- 边缘设备的资源限制,由于内存和计算开销,挑战了传统的持续学习方法.
- 基于memristor的横杆架构提供了节能计算在内存,但其精度较低,阻碍其用于用于连续学习的精确重量调制.
研究的目的:
- 为资源有限的边缘设备提出一种新的持续学习机制.
- 在持续学习中克服低精度memristor设备的局限性.
- 为了减少与边缘设备上的持续学习相关的内存开销和能量消耗.
主要方法:
- 引入了概率的元可塑性,这是一个调节权重更新概率而不是大小的机制.
- 整合了概率性元可塑性到一个使用低精度memristor权重的尖端神经网络中.
- 评估了持续学习基准模型.
主要成果:
- 概率性超塑性实现了与使用高精度权重的最先进的持续学习模型可比的性能.
- 拟议的机制消除了对辅助高精度内存的需求.
- 与辅助内存解决方案相比,对额外参数的内存降低了 ~ 67%,在参数更新期间的能量降低了 ~ 60 倍.
结论:
- 概率性元可塑性为低精度新兴硬件的边缘设备上节能持续学习提供了有效的解决方案.
- 该方法显著减少了内存和能源开销,使先进的AI功能在资源有限的平台上更可行.
- 这项工作为利用memristor技术的实用,低功耗的持续学习系统铺平了道路.
相关概念视频
01:25Long-term Potentiation
2.7K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.7K
01:01Neuroplasticity
298
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
298
01:03Long-term Depression
2.5K
Long-term depression, or LTD, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTD is the process of synaptic weakening that occurs over time between pre and postsynaptic neuronal connections. The synaptic weakening of LTD works in opposition to synaptic strengthening by long-term potentiation (LTP) and together are the main mechanisms that underlie learning and memory.
Calcium Ion Concentration Mechanism
If over...
Calcium Ion Concentration Mechanism
If over...
2.5K
01:19The Role of Ion Channels in Neuronal Computation
3.1K
A postsynaptic neuron usually receives numerous impulses from several other presynaptic neurons. The axon hillock of the postsynaptic neuron integrates all these signals and determines the likelihood of firing an action potential.
Sometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron. However, multiple presynaptic inputs must often create EPSPs around the same time for the postsynaptic neuron to be sufficiently depolarized to fire an action potential....
Sometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron. However, multiple presynaptic inputs must often create EPSPs around the same time for the postsynaptic neuron to be sufficiently depolarized to fire an action potential....
3.1K