在分布回归中对变压器的概括分析
1School of Mathematics and Statistics, University of Sydney, Sydney, NSW 2006, Australia peilin.liu@sydney.edu.au.
Neural computation
|November 18, 2024
概括
本研究介绍了变压器学习的数学框架,解释了它们的注意力机制如何压缩数据. 这为大型语言模型 (LLM) 中的高效技术提供了理论支持.
科学领域:
- 深度学习 (Deep Learning) 是一种深度学习.
- 人工智能的人工智能
- 机器学习理论机器学习理论
背景情况:
- 变压器模型是深度学习的关键,诸如参数效率微调等技术可以提高性能.
- 现有的变压器成功的策略缺乏严格的数学理论支持.
研究的目的:
- 开发一个理论框架,以了解变压器机制和相关技术.
- 通过数学来制定注意力机制,并分析变压器的能力.
主要方法:
- 提出了一个基于分布回归的变压器学习框架.
- 介绍了注意力机制作为"注意力运算符"的数学公式.
- 连接了两个阶段的采样过程与自然语言处理.
主要成果:
- 证明了注意力操作员使变压器能够将分布压缩成信息保存函数表示.
- 与CNN和FCN相比,显示的变压器在学习复杂的函数方面具有更好的能力.
- 从分布回归框架中得出一个受约束的概括.
结论:
- 理论结果为转换器的机制及其在大型语言模型 (LLM) 中的应用提供了洞察力.
- 在新型分析框架内提供了诸如即时调整,参数效率微调和高效缩放等技术的理论解释.
相关概念视频
Transformers in Distribution System
98
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
98
Transformers with Off-Nominal Turns Ratios
139
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
139
Three-Winding Transformers
200
Three identical single-phase transformers can be configured to form a three-phase transformer connection, which involves high-voltage and low-voltage windings. The high-voltage windings are denoted by capital letters A-B-C, while the low-voltage windings are labeled with lowercase letters a-b-c, representing their respective phases. This notation helps distinguish between the high and low voltage sides of the transformer.
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
In the per-unit equivalent circuit of a grounded Y-Y three-phase...
200
Types Of Transformers
948
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
948
Energy Losses in Transformers
834
In an ideal transformer, it is assumed that there are no energy losses, and, hence, all the power at the primary winding is transferred to the secondary winding. However, in reality, the transformers always have some energy losses, and, hence, the output power obtained at the secondary winding is less than the input power at the primary winding due to energy losses.
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
There are four main reasons for energy losses in transformers.
The first cause can be the high resistance of the...
834
Equivalent Circuits for Practical Transformers
397
The practical equivalent circuits of single-phase two-winding transformers exhibit significant deviations from their idealized versions due to the inherent properties of winding resistance and finite core permeability. These properties result in real and reactive power losses, affecting the transformer's performance. Understanding these deviations is crucial for designing more efficient transformers.
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
In a practical transformer, each winding exhibits resistance and leakage reactance. The...
397


