SERE:探索自主监督变压器的功能自我关系
IEEE transactions on pattern analysis and machine intelligence
|August 30, 2023
概括
这项研究引入了一种新的自我监督学习方法,称为视觉转换器 (ViT) 的特征自我关系 (SERE). SERE 增强了 ViT 的功能.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 自主监督学习对于卷积神经网络 (CNN) 在视觉任务中是有效的.
- 视觉转换器 (ViT) 通过空间自我注意和前网络提供强大的表示能力.
- 现有的ViT自我监督方法经常适应CNN策略,忽视ViT的独特特性.
研究的目的:
- 为视觉转化器 (ViT) 开发一种针对视觉转化器 (ViT) 的新型自主监督学习方法.
- 利用ViT在空间和道维度上的固有关系建模能力.
- 提高ViT在下游视觉任务中的代表力.
主要方法:
- 引入了SELf-RElation (SERE) 功能,这是ViTs的新型自我监督学习策略.
- 在训练的特征中利用空间和道自我关系,超越实例级别的歧视.
- 专注于增强特定于ViT架构的关系建模属性.
主要成果:
- SERE显著提高了ViTs的代表性学习能力.
- 拟议的方法在多个下游视觉任务中证明了稳定的性能提升.
- 基于自我关系的学习有效地利用了ViT的架构优势.
结论:
- 功能SELf-RElation (SERE) 提供了一个特定于ViT的自我监督学习方法.
- 这种方法增强了ViT的关系建模,导致了优越的特征表示.
- 在视觉转换器中,SERE为推进自我监督学习提供了一个有希望的方向.
相关概念视频
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
The Ideal Transformer
423
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
423
Three-Winding Transformers
258
Three identical single-phase transformers can be configured to form a three-phase transformer connection, which involves high-voltage and low-voltage windings. The high-voltage windings are denoted by capital letters A-B-C, while the low-voltage windings are labeled with lowercase letters a-b-c, representing their respective phases. This notation helps distinguish between the high and low voltage sides of the transformer.
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
258
Equivalent Circuits for Practical Transformers
464
The practical equivalent circuits of single-phase two-winding transformers exhibit significant deviations from their idealized versions due to the inherent properties of winding resistance and finite core permeability. These properties result in real and reactive power losses, affecting the transformer's performance. Understanding these deviations is crucial for designing more efficient transformers.
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
464
Transformers with Off-Nominal Turns Ratios
176
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
176


