相关实验视频
Updated: Jan 17, 2026

04:48
Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
Published on: July 5, 2024
736
I&S-ViT:一个包容和稳定的方法,用于培训后的ViT量化
IEEE transactions on pattern analysis and machine intelligence
|September 16, 2025
概括
本研究介绍了I&S-ViT,这是一种用于视觉转换器 (ViT) 的培训后量化 (PTQ) 的新方法. 它在低位场景中显著降低了性能下降,使ViT在工业用途中更高效.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 视觉变压器 (ViTs) 提供可扩展的性能,但具有高的计算成本,限制了工业采用.
- 训练后量化 (PTQ) 降低了ViT成本,但往往导致性能降低,特别是在较低的比特宽度.
研究的目的:
- 开发一种新的方法,I&S-ViT,用于视觉转换器的包容性和稳定的培训后量化.
- 解决PTQ期间在Softmax后激活中的量子化低效率以及在LayerNorm后激活中的崎损失格局.
主要方法:
- 引入了一个shift-uniform-log2量化器 (SULQ),用于改进后Softmax激活的域表示和分布近似.
- 开发了一个三阶段的顺优化策略 (SOS),结合了通道智能和层智能定量化,以稳定地学习后LayerNorm激活.
主要成果:
- I&S-ViT在ViT的现有PTQ方法中表现出优越的性能,特别是在低位量化场景中.
- 对W3A3 ViT-B实现了50.68%的显著性能改善,展示了该方法的有效性.
结论:
- I&S-ViT有效地减轻了低位量子化ViT的性能损失,提高了它们在工业应用中的可行性.
- 拟议的SULQ和SOS组件为视觉变压器提供了一个强大的方法,以实现包容性和稳定的PTQ.
相关概念视频
Transformers with Off-Nominal Turns Ratios
523
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
523
Types Of Transformers
1.4K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.4K
The Ideal Transformer
1.4K
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
1.4K
Energy Losses in Transformers
1.3K
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
1.3K
Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
Improving Translational Accuracy
3.6K
3.6K
