相关实验视频
Updated: May 20, 2025

04:48
Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
Published on: July 5, 2024
335
在视觉转换器中,逐步细到粗重建用于准确的低位训练后量子化
Rui Ding1, Liang Yong1, Sihuan Zhao1
1School of Microelectronics and Communication Engineering, Chongqing University, Chongqing, 401331, China.
概括
这项研究引入了渐进微粗重建 (PFCR) 来增强视觉转换器 (ViT) 的低位量化. PFCR显著提高了性能,在量子化ViTs中实现了最先进的精度.
科学领域:
- 计算机视觉 计算机视觉
- 深度学习模型压缩压缩
背景情况:
- 训练后量化 (PTQ) 对于压缩视觉变压器 (ViTs) 是有效的,但通常会导致低位宽度的性能下降.
- 在PTQ中现有的重建方法使用固定的颗粒度,未能利用渐进关系来实现最佳的ViT压缩.
研究的目的:
- 提出一种新的渐进细粗重建 (PFCR) 方法,以提高低位量子化视觉变换器的准确性.
- 引入逐步优化策略 (POS),以促进培训和进一步提高模型性能.
主要方法:
- PFCR重建细粒度单元 (多头自我注意,多层感知子模块和快捷方式),并逐渐将它们组合起来进行更粗的颗粒度重建.
- 使用逐步优化策略 (POS) 来应对与PFCR相关的培训挑战.
主要成果:
- 拟议的PFCR方法在ImageNet数据集上实现了对低位量子化ViT的最先进的Top-1准确性.
- 具体来说,3位量子化ViT-B使用PTQ与PFCR实现了75.61%的Top-1精度.
- 对COCO数据集的评估证明了该方法在对象检测和实例细分方面的有效性和通用性.
结论:
- PFCR在低位量子化视觉转换器的准确性方面提供了显著的改进.
- 该方法在各种计算机视觉任务中表现出强大的性能和概括性.
相关概念视频
Reconstruction of Signal using Interpolation
152
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
152
Transformers with Off-Nominal Turns Ratios
127
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
127
Upsampling
178
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
178
Downsampling
116
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
116
Types Of Transformers
937
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
937
Linear Approximation in Frequency Domain
80
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
80

