免费培训的变压器架构搜索使用零成本代理指导进化
概括
变压器架构搜索 (TAS) 用T-Razor进行了优化,这是一种使用突触多样性和突出性的新方法. 这种方法显著提高了搜索效率,降低了高性能变压器模型的计算成本.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 设计高性能变压器架构是复杂和耗时的.
- 对于神经架构搜索 (NAS) 的现有无培训的代理方法在变压器搜索空间中显示出很差的概括性.
- 需要高效和有效的变压器架构搜索 (TAS) 方法.
研究的目的:
- 为变压器架构搜索 (TAS) 开发一种高效的无培训方法.
- 引入一种与变压器性能相关的新型评估指标.
- 为了提高自动化变压器架构设计的效率和有效性.
主要方法:
- 引入了TRansformer架构的搜索与Zero-cost pRoxy引导进化 (T-Razor).
- 基于多头自我注意力 (MSA) 的突触多样性和多层感知器 (MLP) 的突出性提出的DSS++指标.
- 使用DSS++指导的区块智能进化搜索来优化变压器架构.
主要成果:
- 在四个搜索空间中,T-Razor实现了与最先进的变压器架构相比的竞争性性能.
- 在搜索效率方面取得了显著的改进,将GPU日数从24天以上减少到0.4.4以下.
- 使用T-Razor搜索的变压器在NLP任务的GLUE基准上取得了竞争性结果.
结论:
- T-Razor为无需培训的变压器架构搜索 (TAS) 提供了一个高效和有效的解决方案.
- DSS++指标有效地根据块级属性评估变压器,增强搜索功能.
- 这项工作为无培训的TAS和自动化变压器设计提供了宝贵的见解.
相关概念视频
Energy Losses in Transformers
869
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
869
The Ideal Transformer
391
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
391
Equivalent Circuits for Practical Transformers
420
The practical equivalent circuits of single-phase two-winding transformers exhibit significant deviations from their idealized versions due to the inherent properties of winding resistance and finite core permeability. These properties result in real and reactive power losses, affecting the transformer's performance. Understanding these deviations is crucial for designing more efficient transformers.
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
420
Types Of Transformers
974
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
974
Transformers in Distribution System
102
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
102
Transformers with Off-Nominal Turns Ratios
152
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
152


