关于完全连接层的能源复杂性
Jiří Šíma1, Jérémie Cabessa2, Petra Vidnerová1
1Institute of Computer Science of the Czech Academy of Sciences, Pod Vodárenskou věží 271/2, Prague 8, 182 00, Czechia.
概括
深度神经网络 (DNN) 消耗大量的能量. 本研究分析了完全连接的层的能量复杂性,为移动设备中高效的硬件部署建立了最佳的二次边界.
科学领域:
- 计算机科学 计算机科学
- 电气工程 电气工程
- 人工智能的人工智能
背景情况:
- 深度神经网络 (DNN) 面临着由于其不断增长的大小而不断增加的能源需求.
- 对于在低功耗移动设备上部署DNN的硬件实现至关重要.
- 之前的工作建立了一个验证的,硬件独立的能量复杂性模型,用于卷积神经网络 (CNN).
研究的目的:
- 从理论上分析DNN中完全连接的层的能量复杂性.
- 为了确定最佳的能量复杂性,考虑DRAM和Buffer之间的内存传输.
- 为了验证Simba和Eyeriss硬件的发现.
主要方法:
- 对于完全连接层计算的能量复杂性的理论分析.
- 确定能源复杂性的一般下限.
- 呈现两个数据流来推导能源成本的上限.
- 使用线性编程的弱二元定理进行优化.
- 在Simba和Eyeriss硬件上的实验验证.
主要成果:
- 完全连接层的能源复杂性的一般下限被确定.
- 分析了两个数据流,产生了能源成本的上限.
- 为分区缓冲内存证明了最佳的二次能量复杂性.
- 实验验证证证实了异常最优的二次能量复杂性.
结论:
- 该研究为完全连接的层建立了最佳的二次能量复杂性.
- 这一发现对于在低功耗硬件上高效部署大型DNN至关重要.
- 结果有助于开发节能的人工智能硬件.
相关概念视频
Energy Associated With a Charge Distribution
1.5K
The work done to bring a charge through a distance r is given by the potential difference between the initial and the final position. To assemble a collection of point charges, the total work done can be expressed in terms of the product of each pair of charges divided by their separation distance, defined with respect to a suitable origin. Solving this expression gives the energy stored in a point charge distribution.
1.5K
Circuit Terminology
636
An electrical network is a system composed of interconnected elements, such as resistors, capacitors, inductors, and voltage or current sources. Unlike a circuit, an electrical network does not necessarily form a closed path. In other words, while all circuits can be considered networks due to their interconnected nature, not every network qualifies as a circuit.
A circuit, on the other hand, is also an interconnected system of electrical elements but must contain one or more closed paths.
A circuit, on the other hand, is also an interconnected system of electrical elements but must contain one or more closed paths.
636
Entropy Change in Reversible Processes
2.5K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
2.5K
Energy Losses in Transformers
862
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
862
Equivalent Capacitance
333
From the study of resistive circuits, it is understood that employing a series-parallel combination serves as an effective strategy for simplifying circuits. Capacitors can be arranged within a circuit in one of two ways: a series configuration or a parallel configuration. The way these capacitors are connected to a battery will influence both the potential drop across each individual capacitor and the size of the charge that each capacitor can store. This is determined by the specific type of...
333
Complex Power
402
Power engineers have introduced the concept of complex power to determine the cumulative effect of parallel loads. This idea plays a crucial role in power analysis because it encompasses all the details related to the power consumed by a specific load.
Complex power is defined as the multiplication of the voltage and the complex conjugate of the current. The magnitude of this power, known as apparent power, is measured in volt-amperes (VA). Notably, the angle of the complex power equates to the...
Complex power is defined as the multiplication of the voltage and the complex conjugate of the current. The magnitude of this power, known as apparent power, is measured in volt-amperes (VA). Notably, the angle of the complex power equates to the...
402


