PSAQ-ViT V2:面向视觉变压器的准确和一般的无数据量化
IEEE transactions on neural networks and learning systems
|August 14, 2023
概括
PSAQ-ViT V2 增强了视觉转换器 (ViT) 的无数据量化,使用适应性教师-学生策略. 这种方法可以提高敏感数据应用的准确性,而不需要真实数据.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 模型压缩压缩模型
背景情况:
- 无数据量化解决了模型压缩中的隐私问题,这对于敏感数据至关重要.
- 像PSAQ-ViT这样的现有方法开创了视觉转换器 (ViT) 的无数据量化,使用补丁相似性.
- 对于ViT,需要更准确,更普遍的无数据量化技术.
研究的目的:
- 推出PSAQ-ViT V2,这是视觉转换器 (ViT) 的一个先进的无数据量化框架.
- 为了提高ViT量子化的准确性和普遍性,而无需访问真实世界的数据.
- 为各种视觉任务和模型提供适用于无数据量化的可靠基准.
主要方法:
- 介绍了针对生成样本和量子化学生模型循环演变的适应性教师-学生策略.
- 使用来自PSAQ-ViT的补丁相似度指标,由完全精确的教师模型监督.
- 雇员任务和模型独立的预先信息,消除了辅助类别指导的需要.
主要成果:
- 在图像分类,对象检测和语义细分任务中,PSAQ-ViT V2取得了具有竞争力的结果.
- 在Swin-S骨干上表现出强的性能:在ImageNet上82.13顶级精度,在COCO上50.9盒AP,在ADE20K上47.2mIoU.
- 通过天真的量子化策略,在没有实际数据访问的情况下,始终实现高精度.
结论:
- PSAQ-ViT V2为ViT提供了对ViT的准确和一般无数据定量化的重大进展.
- 该框架显示了作为现实应用程序的实际解决方案的潜力,这些应用程序具有敏感数据限制.
- 发布的代码和竞争结果使PSAQ-ViT V2成为未来研究的强大基准.
相关概念视频
Transformers with Off-Nominal Turns Ratios
176
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
176
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
Transformers in Distribution System
124
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
124
Energy Losses in Transformers
902
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
902
The Ideal Transformer
426
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
426
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K


