GradToken:解令牌与类意识的梯度,用于视觉解释变压器网络的解令牌
Lin Cheng1, Yanjie Liang2, Yang Lu1
1Fujian Key Laboratory of Sensing and Computing for Smart City, School of Informatics, Xiamen University, Xiamen 361005, China.
概括
研究人员开发了GradToken,这是对变压器网络的一种新的视觉解释方法. 这种方法有效地解决了注意力重量的语义合,提高了变压器模型在各种应用中的可解释性和可靠性.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 变压器网络越来越多地被用于各种AI领域,包括计算机视觉和自然语言处理.
- 解释变压器决策对于增强对现实应用的信任和理解至关重要.
- 由于结构上的差异,对卷积神经网络的现有解释方法不能直接适用于变压器架构.
研究的目的:
- 为了应对设计有效的解释方法的挑战,具体用于变压器网络.
- 为了克服变压器注意力权重矩阵中的语义合问题,这阻碍了特定类别的解释.
- 提出一种新的方法,用于生成变压器预测的视觉解释.
主要方法:
- 分析了变压器注意力权重矩阵中的语义合问题.
- 提出GradToken,一种基于梯度解的令牌相关性方法.
- 利用类意识梯度来解语义,并利用符号关系来生成相关性地图.
主要成果:
- GradToken有效地解了类令牌中的纠语义,使类别特定的解释成为可能.
- 生成的相关性地图准确地突出了目标地区,提高了视觉解释的质量.
- 广泛的实验验证了该方法的有效性和可靠性.
结论:
- GradToken提供了一种特定且有效的解决方案,以视觉解释变压器预测.
- 该方法通过生成聚焦和可靠的相关性地图来提高变压器的解释性.
- 这项工作有助于在应用变压器模型方面建立更大的信任和理解.
相关概念视频
Types Of Transformers
948
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
948
Energy Losses in Transformers
834
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
834
Transformers in Distribution System
98
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
98
Equivalent Circuits for Practical Transformers
397
The practical equivalent circuits of single-phase two-winding transformers exhibit significant deviations from their idealized versions due to the inherent properties of winding resistance and finite core permeability. These properties result in real and reactive power losses, affecting the transformer's performance. Understanding these deviations is crucial for designing more efficient transformers.
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
397
The Ideal Transformer
356
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
356
Transformers with Off-Nominal Turns Ratios
139
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
139


