使用模拟内存进行等效精度加速神经网络训练
Stefano Ambrogio1, Pritish Narayanan1, Hsinyu Tsai1
1IBM Research-Almaden, San Jose, CA, USA.
Nature
|June 8, 2018
概括
这项研究引入了一种新的混合硬件和软件方法,用于更快,更节能的神经网络训练. 这种新方法实现了与软件培训相美的分类精度,显著提高了计算效率.
科学领域:
- 材料科学
- 计算机科学
- 电气工程
背景情况:
- 由于内存和处理器之间的数据传输, 神经网络的训练需要大量的能量.
- 模拟非易失性内存为现场训练提供了潜力,但其准确性受到限制.
- 现有的模拟内存硬件显示动态范围不足和重量更新不对称.
研究的目的:
- 开发一个混合的硬件和软件系统,用于高效的神经网络训练.
- 通过使用非易失性记忆来克服现场训练的精度限制.
- 提高神经网络加速器的计算能效和吞吐量.
主要方法:
- 实现多达204,900个突触的混合硬件和软件神经网络.
- 用于长期存储的相变内存和用于近线性更新的挥发性电容.
- 在重量数据传输中使用"极性逆转"来减轻装置变化.
主要成果:
- 在标准机器学习数据集 (MNIST,CIFAR-10/100) 上实现与基于软件的训练相当的概括精度.
- 已证明计算能效为280.65亿次/秒/瓦.
- 每个面积的计算吞吐量超过当前图形处理单元的两个数量级.
结论:
- 开发的系统为神经网络提供了快速和节能硬件加速器的可行途径.
- 这种方法对于完全连接的神经网络层特别有效.
- 这些发现为下一代人工智能硬件铺平了道路.
相关概念视频
Improving Translational Accuracy
15.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.0K
Equivalent Capacitance
2.2K
Multiple capacitors can be connected in a circuit in series or parallel configuration. When the capacitor combination is connected to a battery, the potential drop across each capacitor and the magnitude of charge stored in the individual capacitor depends on the type of the connection. The capacitor combination is replaced by a single equivalent capacitor that stores the same amount of charge as the combination for a given potential difference.
The following strategies are adopted to calculate...
The following strategies are adopted to calculate...
2.2K
Pharmaceutical Equivalents
205
As defined by regulatory standards, pharmaceutical equivalents require generic drug products to have identical dosage forms and chemically identical active pharmaceutical ingredients (APIs). They must adhere to compendial or applicable standards for potency, content uniformity, disintegration times, and dissolution rates. In the case of modified-release dosage forms, variations in drug content are permissible as long as the delivered amount remains consistent with the innovator drug product.
205
Uncertainty in Measurement: Accuracy and Precision
102.9K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
102.9K
Accelerators
292
Accelerators in concrete serve as admixtures to speed up the hardening process, enabling the concrete to achieve early strength faster. Although accelerators do not necessarily impact the time it takes concrete to set, they reduce this time in practice. A common accelerator is calcium chloride, which is particularly useful for hastening early strength development in cold weather or for rapid repair jobs that require quick heat generation after mixing.
The effectiveness of calcium chloride can...
The effectiveness of calcium chloride can...
292
Protein Networks
4.6K
An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
4.6K


