数字串行 DA 基于固定点 RNN:一种统一的方法来提高架构效率
概括
本研究引入了新的,高效的长期短期记忆 (LSTM) 循环神经网络 (RNN) 的新型架构,使用二的补数数字-序列分布式算法. 这些设计显著提高了嵌入式AI应用的面积和能源效率.
科学领域:
- 计算机工程 计算机工程
- 人工智能的人工智能
- 嵌入式系统 嵌入式系统
背景情况:
- 将先进的人工智能 (AI) 神经网络模型集成到资源有限的嵌入式和移动系统中是一个关键的挑战.
- 在中开发紧,节能的神经网络模型对于优化这些系统的性能至关重要.
- 长期短期记忆 (LSTM) 循环神经网络 (RNN) 是强大的,但计算密集型,需要进行架构增强以提高效率.
研究的目的:
- 提出一种统一的方法来提高LSTM RNNs的架构效率.
- 在LSTM RNNs中引入两种新型结构 (结构I和结构II) 进行高效的计算.
- 用固定点定量化模型验证语音识别任务的拟议结构.
主要方法:
- 开发了两种新的LSTM RNN结构 (I和II),利用两者的补充 (TC) 数字序列分布式算术 (DSDA) 技术.
- 使用TC DSDA. 制定了块循环矩阵向量乘法 (MVM) 和元素智能乘法 (EWM).
- 引入了新的部分产品生成器 (PPG) 和部分产品选择器 (PPS),兼容签名和未签名的数字,并采用了定点 (FxP) 训练程序来定量化LSTM RNNs.
主要成果:
- 结构I使用非管道式MVM与转移积累单元 (SAU),而结构II使用深度管道式MVM与转移添加树 (SAT).
- 28纳米FDSOI技术的合成后结果显示了显著的改进:结构I提供39.87% (面积) 和30.95% (能量) 的效率增长.
- 结构II表现出更大的收益,实现95.63% (面积) 和91.18% (能源) 的效率提高,与以前的工作相比,模型大小为128美元×128美元.
结论:
- 拟议的基于TC DSDA的结构为LSTM RNNs的架构效率提供了实质性的改进.
- 这些结构对于在中实现量子化LSTM RNN非常有效,特别适用于语音识别等应用.
- 新的PPG和PPS组件有助于提议设计的整体效率和灵活性.
更多相关视频
05:49Author Spotlight: Analgesic Effect of Tuina on Rat Models with Compression of the Dorsal Root Ganglion Pain
Published on: July 14, 2023
1.4K
07:16Author Spotlight: Advancements in DNA Nanosensors – Addressing Sensitivity and Selectivity Challenges in Molecular Detection
Published on: February 9, 2024
944
相关概念视频
Basic Discrete Time Signals
202
The unit step sequence is defined as 1 for zero and positive values of the integer n. This sequence can be graphically displayed using a set of eight sample points, showing a step function starting from n=0 and remaining constant thereafter.
The unit impulse or sample sequence is mathematically expressed as zero for all n values except at n=0, where it is one. The unit impulse sequence, denoted by δ(n), is the first difference of the unit step sequence, while the unit step sequence u(n) is...
The unit impulse or sample sequence is mathematically expressed as zero for all n values except at n=0, where it is one. The unit impulse sequence, denoted by δ(n), is the first difference of the unit step sequence, while the unit step sequence u(n) is...
202
Numerical Calculations
349
In engineering applications, the representation of the numerical value is critical. Presenting or reporting the answer is one of the essential parts of engineering practices. Numerical calculations are performed using handheld calculators or computers since numerically accurate answers are always preferred.
The solution to a problem is obtained using different methods. While manually solving algebraic symbols is one of the most common methods, the graphical method is often preferred. Computers...
The solution to a problem is obtained using different methods. While manually solving algebraic symbols is one of the most common methods, the graphical method is often preferred. Computers...
349
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
47
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
47
Fast Fourier Transform
296
The Fast Fourier Transform (FFT) is a computational algorithm designed to compute the Discrete Fourier Transform (DFT) efficiently. By breaking down the calculations into smaller, manageable sections, the FFT significantly reduces the computational complexity involved. Direct computation of an N-point DFT requires N2 complex multiplications, whereas the FFT algorithm needs only (N/2)log2N multiplications, offering a much faster performance.
The computational efficiency of the FFT becomes...
The computational efficiency of the FFT becomes...
296
Design Example
323
The innovation of touch-tone telephony revolutionized the telecommunications industry by replacing the traditional rotary dial with a dual-tone multi-frequency (DTMF) signaling system. This system uses a matrix-style keypad with buttons arranged in four rows and three columns, creating 12 distinct signals each assigned to a pair of frequencies. Each button press results in a simultaneous generation of two sinusoidal tones – one from a low-frequency group (697 to 941 Hz) and one from a...
323
Discrete-Time Fourier Series
246
The Discrete-Time Fourier Series (DTFS) is a fundamental concept in signal processing, serving as the discrete-time counterpart to the continuous-time Fourier series. It allows for the representation and analysis of discrete-time periodic signals in terms of their frequency components. Unlike its continuous counterpart, which utilizes integrals, the calculation of DTFS expansion coefficients involves summations due to the discrete nature of the signal.
For a discrete-time periodic signal x[n]...
For a discrete-time periodic signal x[n]...
246
