神经机器翻译中变压器的直角位置表示
Yue Zhao1, Qinghong Zhang2, Shuhan Zhou2
1School of Marxism, Northeastern University, Shenyang, China.
PloS one
|October 15, 2025
概括
一个新的直角固定维度位置表示 (OPR) 增强了用于神经机器翻译的变压器模型. 这种无参数的方法提高了序列的理解,并优于现有的定位编码技术.
科学领域:
- 自然语言处理自然语言处理.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 变压器架构主导神经机器翻译 (NMT) 由于其捕获远程依赖的能力.
- 变压器中的自我注意机制对位置不敏感,因此需要对序列顺序进行有效的位置编码.
- 现有的定位编码方法,如正弦-小和可学习嵌入有局限性.
研究的目的:
- 为基于变压器的NMT引入一种新的正交固定维度位置表示 (OPR).
- 为了增强在序列内令牌位置的区分,而无需添加参数或计算开销.
- 通过提供卓越的顺序上下文来提高NMT模型的性能.
主要方法:
- 提出了一种新的直角固定维度位置表示 (OPR).
- 在NMT任务的变压器模型中实现了OPR.
- 在多个标准NMT数据集上进行实验.
- 使用BLEU和COMET分数进行绩效评估.
主要成果:
- OPR的性能始终优于基线定位编码方法 (正弦-正弦,可学习嵌入).
- 在所有测试的语言对中,在BLEU和COMET分数中取得了显著的改善.
- 当与相对位置编码 (RPR) 结合使用时,OPR显示了增强的性能.
结论:
- OPR是一种高效且有效的NMT位置编码方法.
- OPR成功地模拟了绝对和相对位置关系,这对于理解序列至关重要.
- 拟议的OPR为变压器模型提供了一个无参数和计算成本低廉的进步.
相关概念视频
Transformers with Off-Nominal Turns Ratios
519
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
519
Types Of Transformers
1.4K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.4K
Position Vectors
1.8K
A position vector is a fundamental concept in mathematics that helps determine the position of one point with respect to another point in space. It is a vector that describes the direction and distance between two points. Position vectors are highly useful in the field of math and science, as they help represent spatial relationships and make calculations easier.
For instance, we want to locate a point P(x, y, z) relative to the origin of coordinates O. In that case, we can define a position...
For instance, we want to locate a point P(x, y, z) relative to the origin of coordinates O. In that case, we can define a position...
1.8K
Per-Unit Sequence Models
431
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
431
Orthogonal Trajectories
6
Orthogonal trajectories describe the geometric relationship between two families of curves that intersect each other at right angles. One illustrative case involves a family of parabolas that open sideways along the x-axis. These curves share a common shape but differ by a scaling parameter, resulting in a set of curves that all pass through the origin and widen at different rates.Determining Orthogonal TrajectoriesTo identify the orthogonal trajectories for these parabolas, the first step...
6
Transformers
1.7K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.7K
