在14nm模拟AI推理芯片上演示基于变压器的ALBERT模型
An Chen1, Stefano Ambrogio2, Pritish Narayanan2
1IBM Research - Almaden, San Jose, CA, USA. chenan@us.ibm.com.
Nature communications
|September 30, 2025
概括
这项研究展示了在模拟硬件上使用相变内存的大型变压器模型,尽管硬件不完美,但实现了接近ISO的准确性. 硬件意识微调和漂移补偿技术显著提高了性能和稳定性.
科学领域:
- 人工智能的人工智能
- 硬件加速器 硬件加速器
- 神经形态计算是一种神经形态计算.
背景情况:
- 变压器模型是计算密集型的,限制了它们在边缘设备上的部署.
- 模拟计算为节能AI推断提供了一个潜在的解决方案.
- 阶段变换存储器 (PCM) 是对模拟硬件的一种有希望的非易失性存储器技术.
研究的目的:
- 在14nm模拟推理芯片上展示一个大型变压器模型 (变压器的轻双向编码器表示).
- 为了评估模拟实现的性能和准确性与浮点参考.
- 调查和减轻诸如重量编程错误,硬件缺陷和设备漂移等挑战.
主要方法:
- 将Lite Transformer模型的710万个模拟重量映射到基于PCM的单个模拟芯片上的2830万个设备上.
- 编程设备导电量以高精度表示模型重量.
- 使用硬件意识微调技术来优化模拟芯片上的模型性能.
- 实施基于重新校准的漂移补偿策略,以解决导电漂移问题.
主要成果:
- 在通用语言理解评估基准上实现了接近iso精度,平均精度仅为1.8%低于浮点参考值.
- 硬件意识微调将平均硬件精度提高了4.4%.
- 漂移补偿将30天的精度损失从5%降低到1%以下.
结论:
- 带有PCM的模拟硬件可以成功地实现高精度的大型变压器模型.
- 硬件意识培训和漂移补偿对于强大的模拟AI系统至关重要.
- 这项工作是朝着对模拟硬件进行节能,大规模AI推断迈出的重要一步.
相关概念视频
01:28Equivalent Circuits for Practical Transformers
1.4K
The practical equivalent circuits of single-phase two-winding transformers exhibit significant deviations from their idealized versions due to the inherent properties of winding resistance and finite core permeability. These properties result in real and reactive power losses, affecting the transformer's performance. Understanding these deviations is crucial for designing more efficient transformers.
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
1.4K
01:25Transformers with Off-Nominal Turns Ratios
523
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
523
01:26The Ideal Transformer
1.4K
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
1.4K
01:17Ampere-Maxwell's Law: Problem-Solving
1.1K
A parallel-plate capacitor with capacitance C, whose plates have area A and separation distance d, is connected to a resistor R and a battery of voltage V. The current starts to flow at t = 0. What is the displacement current between the capacitor plates at time t? From the properties of the capacitor, what is the corresponding real current?
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of the...
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of the...
1.1K
01:16Types Of Transformers
1.4K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.4K
01:11Source Transformation for AC Circuits
1.0K
The process of source transformation in the frequency domain entails the conversion of a voltage source, positioned in series with an impedance, into a current source that is parallel to an impedance, or the other way around. It is essential to maintain the following relationships while transitioning from one source type to another.
1.0K