有效的扭曲最小化的层次修剪
概括
这项研究引入了一种新的深度神经网络 (DNN) 的训练后修剪框架,可以最大限度地减少输出扭曲. 该方法在各种模型和任务中实现了浮点运算 (FLOP) 的显著减少,而不会影响准确性.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 深度神经网络 (DNN) 是计算密集型的,需要高效的模型压缩方法.
- 修剪是一种减少模型大小和复杂性的技术,在优化层次减少和最小化输出扭曲方面经常面临挑战.
研究的目的:
- 开发一个培训后修剪框架,共同优化层级修剪,以尽量减少模型输出扭曲.
- 为了利用输出扭曲的新发现的附加性属性,以实现高效的修剪优化.
主要方法:
- 建议采用训练后修剪框架,优化层级修剪以最大限度地减少模型输出扭曲.
- 在DNN中确定并利用输出扭曲的附加性属性.
- 修剪优化被重新阐述为一个通过动态编程解决的组合式问题,实现线性时间复杂性.
- 采用基于Hessian的泰勒近似来优化扭曲,提高修剪效率.
主要成果:
- 该框架实现了最先进的 (SoTA) 结果,在各种DNN架构 (CNNs,ViTs) 和任务 (图像分类,3D对象检测) 中显著减少了FLOP.
- 例如,在CIFAR-10 (VGG-16) 上减少了多达29.2倍的FLOP,在ImageNet (DeiT-Base) 上减少了2倍的FLOP,没有精度损失.
- 在3D物体检测模型中也观察到显著的FLOP减少,例如CenterPoint (3.89x) 和PVRCNN (3.72x).
结论:
- 拟议的层适应性重量修剪框架是有效和实用的,用于提高模型性能.
- 该方法在不降低准确度的情况下显示了显著的计算节约 (FLOP减少).
- 基于添加性属性的动态编程方法为DNN修剪提供了快速有效的解决方案.
更多相关视频
09:16Author Spotlight: Optimization of Processing Technology for Tiebangchui with Zanba Based on CRITIC Combined with Box-Behnken Response Surface Method
Published on: May 12, 2023
1.2K
13:44Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
Published on: August 30, 2013
43.0K
相关概念视频
01:18Reducing Line Loss
196
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
196
01:20Downsampling
260
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
260