在反复性神经网络的模拟电阻横杆中有效的非线性函数近似
Junyi Yang1, Ruibin Mao2, Mingrui Jiang2
1Department of Electrical Engineering, City University of Hong Kong, Hong Kong SAR, China.
Nature communications
|January 29, 2025
概括
本研究介绍了一种使用模拟内存计算 (IMC) 进行反复神经网络 (RNN) 的新方法. 新方法有效地实现非线性激活函数,提高语音和语言任务的性能.
科学领域:
- 神经形态工程的神经形态工程
- 计算机科学 计算机科学
- 材料科学 材料科学 材料科学
背景情况:
- 模拟内存计算 (IMC) 为特定层提供了节能的深度神经网络 (DNN) 加速.
- 对于语音和自然语言处理至关重要的循环神经网络 (RNN),由于非线性激活函数,在IMC中面临挑战.
- 当前的IMC方法在实现这些非线性方面需要大量的能量和时间.
研究的目的:
- 通过实验证明一个集成的非线性激活函数,并进行道模拟到数字转换 (ADC),以改进内存RNN实现.
- 使用IMC提高RNN在语音识别和自然语言处理任务中的效率.
- 为复杂的神经网络架构克服现有的IMC方法的局限性.
主要方法:
- 利用额外的memristor列来产生预先扭曲的坡道电压,用于近似非线性函数.
- 在内存外围集成了一个道模拟到数字转换器 (ADC).
- 在一个memristive数组中实验性地编程各种非线性函数.
- 模拟了将这种方法集成到RNN中,用于关键字发现和语言建模.
主要成果:
- 通过使用memristive数组成功演示了各种非线性函数的编程.
- 与其他方法相比,在面积效率,能源效率和吞吐量方面取得了显著的改进.
- 验证了内存可编程坡道发生器在去除数字处理开销方面的有效性.
结论:
- 拟议的方法通过高效处理非线性激活函数,显著增强内存RNN实现.
- 这一创新为更强大,更有效的语音和语言处理神经形态硬件铺平了道路.
- 集成式坡道发电机为先进的人工智能应用提供了在面积,能量和速度方面显著的优势.
相关概念视频
Linear Approximation in Frequency Domain
85
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
85
Linear Approximation in Time Domain
60
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
60
Linear Circuits
380
A linear circuit is characterized by its output having a direct proportionality to its input, adhering to the linearity property, which encompasses the principles of homogeneity (scaling) and additivity. Homogeneity dictates that when the input, also referred to as the excitation, is multiplied by a constant factor, the output, known as the response, is correspondingly scaled by the same constant factor. For instance, if the current is multiplied by a constant 'k,' the voltage likewise...
380
Network Function of a Circuit
255
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
255
Equivalent Resistance
380
In circuit analysis, situations often arise where resistors are neither in series nor parallel configurations. To tackle such scenarios, three-terminal equivalent networks like the wye (Y) (Figure 1 (a)) or tee (T) and delta (Δ) (Figure 1 (b)) or pi (π) networks come into play. These networks offer versatile solutions and are frequently encountered in various applications, including three-phase electrical systems, electrical filters, and matching networks.
380
Second-order Op Amp Circuits
233
Implementing second-order low-pass filters in audio systems is crucial in refining audio signals by eliminating undesirable high-frequency noise. These filters typically involve second-order op-amp circuits configured as voltage followers, encompassing two nodes with distinct storage elements.
The analysis of such circuits follows a systematic approach, similar to the second-order RLC circuits. In practical scenarios, bulky inductors are rarely employed due to their size and weight. This means...
The analysis of such circuits follows a systematic approach, similar to the second-order RLC circuits. In practical scenarios, bulky inductors are rarely employed due to their size and weight. This means...
233


