通过减少错误来实现视觉转换器准确的训练后量化
概括
ERQ是一种新的训练后量化方法,通过顺序处理激活和重量量化来减少视觉变压器中的错误. 这种方法显著提高了准确性,超过了现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 模型优化模型优化
背景情况:
- 培训后量化 (PTQ) 对视觉转换器 (ViT) 的有效部署至关重要.
- 现有的PTQ方法与复杂的重量激活相互作用作斗争,导致性能降低.
- 需要使用PTQ技术来最大限度地减少ViTs中的量化错误.
研究的目的:
- 为ViT引入ERQ,一种两步PTQ方法.
- 为了减少来自激活和权重的量子化错误.
- 为了提高量子化ViTs的性能.
主要方法:
- ERQ采用了一种连续的两步方法:激活量化误差减少 (Aqer) 和重量量化误差减少 (Wqer).
- Aqer使用修复参数化初始化和回归来处理激活错误.
- 对于重量错误,Wqer使用双统一量化和代圆形化精细化与斜坡回归.
主要成果:
- ERQ显著减少了ViTs中的量子化误差.
- 该方法在各种ViT模型和任务中展示了卓越的性能.
- 对于W3A4 ViT-S. 的ERQ,其精度比GPTQ提高了36.81%.
结论:
- 通过其连续的两步策略,ERQ有效地减少了ViT中的量化错误.
- 拟议的方法提供了一个强大的解决方案,用于高精度部署量子化ViT.
- ERQ代表了视觉转换器PTQ技术的重大进步.
更多相关视频
05:49Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders
Published on: November 1, 2024
672
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
263
相关概念视频
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
02:07Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
01:21Energy Losses in Transformers
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the copper windings...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the copper windings...
01:25Transformers with Off-Nominal Turns Ratios
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the rated...