基于张量环分解的神经网络压缩
概括
深度神经网络 (DNN) 可以使用张量环 (TR) 分因分解进行压缩. 这种方法可以显著降低内存和计算成本,以便在各种设备上有效部署.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 深度神经网络 (DNN) 提供了高精度,但遭受了大量的内存和计算需求.
- 这些局限性阻碍了在桌面和移动平台等资源有限的设备上部署DNN.
- 低级因子化是网络压缩的一个关键技术,通过分解参数来减少模型大小.
研究的目的:
- 探索张量环 (TR) 分因子对压缩深度神经网络的应用.
- 研究参数张量重塑和TR分解 (TRD) 对模型压缩的影响.
- 开发一个算法,以优化张量重塑和TRD最大参数减少.
主要方法:
- 拟用于神经网络压缩的杆张量环 (TR) 分因. 神经网络压缩.
- 研究了参数张量重塑和TR分解 (TRD) 的影响.
- 开发了一种基于素因子算法的算法,用于最佳的张量重塑和TRD.
- 引入了一种新的树结构和一个自上而下的分割算法,通过优化核心张量执行顺序来最大限度地降低计算复杂性.
主要成果:
- 拟议的算法通过最佳张量重塑和TRD实现最大参数压缩.
- 核心张量器的不同执行顺序显著影响计算复杂性.
- 开发的调度算法有效地减少了计算复杂性.
- 在三种神经网络类型和数据集的广泛实验中,与最先进的低级因子分解方法相比,表现出了更高的性能.
结论:
- 张量环因子化为压缩深度神经网络提供了一种强大的方法.
- 提出的基于素因子算法和张量调度方法显著降低了内存消耗和计算复杂性.
- 这项工作使得在资源有限的设备上更有效地部署DNN.
相关概念视频
01:24Sequence Networks of Rotating Machines
99
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
99
01:27Transformers in Distribution System
102
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
102
01:08Vector Algebra: Method of Components
13.9K
It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
In many applications, the magnitudes and directions of...
13.9K
01:26Cartesian Form for Vector Formulation
631
The Cartesian form for vector formulation is a process to calculate the moment of force using the position and force vectors. The moment of force is defined as the cross-product of these vectors, making it a vector quantity. The Cartesian form of the position and force vectors involves unit vectors, which can be used to express the cross-product in determinant form.
631
01:22Block Diagram Reduction
202
The process of deriving the transfer function of a control system often involves reducing its block diagram to a single block. This simplification can be achieved through a series of strategic operations, including relocating branch points and comparators. These operations preserve the overall function of the system while allowing for easier manipulation and combination of blocks.
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
202
01:25Transformers with Off-Nominal Turns Ratios
151
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
151