基于Tiki-Taka算法的模拟深度学习加速器的保留意识零转移技术
Kyungmi Noh1, Hyunjeong Kwak1, Jeonghoon Son1
1Department of Materials Science and Engineering, Pohang University of Science and Technology, Pohang 37673, Republic of Korea.
Science advances
|June 14, 2024
概括
我们开发了4K级电化学随机存储器 (ECRAM) 阵列,用于模拟神经网络训练. 一种新的技术优化了训练精度,尽管设备保留限制.
科学领域:
- 材料科学 材料科学 材料科学
- 计算机工程 计算机工程
- 人工智能的人工智能
背景情况:
- 电化学随机访问存储器 (ECRAM) 为模拟神经网络加速器提供了潜力.
- 模拟内存设备的非理想特性对高效的神经网络训练提出了挑战.
研究的目的:
- 为模拟神经网络训练制造和描述大型ECRAM交点阵列.
- 研究ECRAM记忆特征对神经网络训练性能的影响.
- 开发一种技术,以提高基于ECRAM的系统的培训准确性.
主要方法:
- 4K级ECRAM交点阵列的制造.
- 一个8x8 ECRAM阵列的电气特性,评估产量和变化.
- 使用具有Tiki-Taka版本2 (TTv2) 算法的ECRAM设备进行实验研究.
- 开发和应用一种保持意识的零转移技术.
主要成果:
- 实现了8x8 ECRAM阵列的100%产量,具有出色的切换特性和较低的变化.
- 在使用TTv2.2进行神经网络训练时,证明了ECRAM阵列的有效性.
- 确定保留特征作为影响训练准确性和可用的体重范围的关键因素.
- 通过使用提议的保留意识零转移技术,展示了提高训练绩效.
结论:
- 大规模的ECRAM阵列对于模拟神经网络训练加速器是可行的.
- 设备保留特征显著影响神经网络训练结果.
- 保持意识的零转移技术有效地优化了限制保留的ECRAM设备的训练性能.
相关概念视频
01:10Reconstruction of Signal using Interpolation
191
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
191
01:25Long-term Potentiation
2.8K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.8K
01:11Sampling Continuous Time Signal
226
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
226