两阶段合作模型压缩培训,用于联合修剪和量子化
Chunxiao Fan1, Jintao Li2, Zhongqian Zhang2
1Key Laboratory of Knowledge Engineering with Big Data, Ministry of Education, Hefei University of Technology, Hefei, 230009, Anhui, China; Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230088, Anhui, China.
概括
这项研究引入了联合神经网络修剪和定量化的两阶段框架. 该方法协同优化了多种压缩技术,降低了模型的复杂性,同时保持了准确性.
科学领域:
- 人工智能的人工智能
- 计算机科学 计算机科学
- 机器学习 机器学习
背景情况:
- 神经网络模型压缩对于减少存储和计算需求至关重要.
- 现有的压缩方法,如修剪和量子化,往往缺乏有效的整合,限制了性能.
- 需要采用统一的方法,才能同时利用各种压缩技术的好处.
研究的目的:
- 提出一个新的两阶段合作培训框架,用于联合修剪和量化.
- 为了实现多个神经网络压缩技术的协同优化.
- 通过整合修剪和量化,提高模型压缩效率和准确性.
主要方法:
- 一个两阶段的框架:协作约束前压缩和训练后压缩改进.
- 统一的约束损失函数用于重量接近量化值.
- 在修剪中为自动化网络结构学习进行稀疏的规范化.
- 代优化以最大限度地减少量化错误并实现2^n量化.
主要成果:
- 网络参数显著减少,在MNIST,CIFAR-10和CIFAR-100数据集上保持了相当大的准确性.
- 在压缩比和精度方面都取得了卓越的效率.
- 该框架成功地整合了修剪和量化,最大限度地减少了不良相互作用.
结论:
- 拟议的框架有效地整合了用于协同模型压缩的修剪和定量化.
- 它提供了一个可行的解决方案,用于开发更高效的神经网络模型,适合硬件实现.
- 这种方法最大限度地减少了压缩技术之间的负面影响,提高了整体性能.
相关概念视频
Reducing Line Loss
338
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
338
Structural Classification of Joints
6.8K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
6.8K
Extraction: Partition and Distribution Coefficients
4.5K
The distribution law or Nernst's distribution law is the law that governs the distribution of a solute between two immiscible solvents. This law, also known as the partition law, states that if a solute is added to the mixture of two immiscible solvents at a constant temperature, the solute is distributed between the two solvents in such a way that the ratio of solute concentrations in the solvents remains constant at equilibrium.
For extracting a solute from an aqueous phase into an...
For extracting a solute from an aqueous phase into an...
4.5K
Compacting Factor test
521
The compacting factor test is a method used to assess the workability of concrete. It is especially suitable for concrete mixes containing aggregates up to one and a half inches in size. This test involves specialized equipment consisting of two truncated cone-shaped hoppers and a cylinder, all with polished interior surfaces to minimize friction.
The procedure begins by placing concrete into the upper hopper without any compaction. Once filled, the bottom door of this hopper is opened,...
The procedure begins by placing concrete into the upper hopper without any compaction. Once filled, the bottom door of this hopper is opened,...
521
Reconstruction of Signal using Interpolation
660
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
660
Functional Classification of Joints
6.5K
Functional Classification of Joints
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
6.5K

