加快大规模视觉变压器,以便在没有微调的情况下进行密集的预测
IEEE transactions on pattern analysis and machine intelligence
|October 25, 2023
概括
这项研究引入了新的方法来加速视觉转换器,用于密集的预测任务,而无需微调. 该方法使用代币集群和重建来提高各种应用程序的效率和性能.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 大规模视觉变压器在密集预测方面取得了最先进的结果,但在计算上昂贵.
- 现有的加速方法主要集中在图像分类上,而不是密集的预测任务.
- 需要高效的视觉转换器架构来进行密集的预测,而无需对特定任务进行微调.
研究的目的:
- 开发一种新的,无微调的方法来加速视觉转换器在密集的预测任务中.
- 引入非参数运算符,降低计算成本,同时保持性能.
- 为了证明在广泛的密集预测应用中提出的方法的多功能性.
主要方法:
- 建议使用token聚类层,通过分组邻近的表示来减少token的数量,从而创建低分辨率的特征地图.
- 变压器层仅适用于这些凝聚的低分辨率令牌,以减少计算.
- 引入了一个令牌重建层,从精细的低分辨率特征中恢复高分辨率表示.
主要成果:
- 提出的方法在六个不同的密集预测任务中取得了有希望和一致的结果.
- 有效性是根据最先进的开放词汇识别方法进行验证的.
- 该方法在密集预测基准上与现有的代表方法相比,显示了显著的计算节约.
结论:
- 开发的代币集群和重建层提供了一种有效的方式来加速视觉变压器,以实现密集的预测.
- 该方法的无微调性质扩大了其在各种计算机视觉任务中的适用性.
- 这项工作是迈向更具计算可行性和高性能视觉变压器模型密集预测的重要一步.
相关概念视频
Vision
53.5K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.5K
Depth Perception and Spatial Vision
681
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
681
Transformers with Off-Nominal Turns Ratios
162
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
162
Improving Translational Accuracy
11.4K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.4K
End Point Prediction: Gran Plot
345
A Gran plot is used to predict the equivalence volume or endpoint of a potentiometric or acid-base titration without reaching the endpoint. Typically, titration data is collected as a function of the titrant's volume up to a point less than the equivalence volume and then transformed into a linear format. The straight line is extended to the x-axis, indicating the necessary titrant volume to achieve the equivalence point.
For potentiometric titration, the Gran plot is created by plotting...
For potentiometric titration, the Gran plot is created by plotting...
345
Visual System
594
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
594


