蒙面通道建模使视觉转换器能够学习更好的语义
Jiayi Chen1, Yanbiao Ma2, Wei Dai1
1School of Telecommunications Engineering, Xidian University, Xi'an 710071, China.
Entropy (Basel, Switzerland)
|August 28, 2025
概括
蒙面通道建模 (MCM) 通过重建通道特征来增强视觉转换器,从而提高语义理解. 这种新的方法在各种下游视觉任务中胜过现有方法.
科学领域:
- 计算机视觉
- 机器学习
- 人工智能
背景情况:
- 视觉转换器 (ViT) 擅长模拟空间环境.
- 蒙面图像建模 (MIM) 是ViT的常见预训练技术,专注于空间重建.
- 现有的MIM方法往往忽视了通道维度中的语义连续性.
研究的目的:
- 引入一种新的预训练模式,即面具道建模 (MCM).
- 通过专注于道语义连续性来增强视觉表示学习.
- 通过道视角来改善图像特征的理解.
主要方法:
- 提出掩盖道建模 (MCM) 预培训模式.
- 使用未掩盖道的上下文信息重建掩盖道特征.
- 使用CLIP图像编码器功能作为增强语义属性的高级重建目标.
主要成果:
- 在下游任务中显著提高了MCM的性能.
- 在现有方法中表现出有效性和优势.
- 通过道语义连续性增强对图像的模型理解.
结论:
- 对于视觉转换者来说,MCM是一种有效的预训练策略.
- 专注于道语义连续性为MIM提供了一个新的方向.
- 提出的方法显示了推进视觉表现学习的巨大潜力.
相关概念视频
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
Transformers with Off-Nominal Turns Ratios
205
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
205
Observational Learning
310
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
310
Modeling and Similitude
328
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
328
Vision
55.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.3K
Masking and Demasking Agents
2.6K
EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
2.6K


