一个纯粹的变压器训练框架在文本赋值图表上
Yu Song1, Haitao Mao1, Jiachen Xiao1
1Michigan State University.
概括
本研究介绍了用变压器 (GSPT) 进行图表序列训练,这是一种用于图表表示学习的新方法. GSPT有效地处理特征和结构异质性,改善跨多种图形数据集的模型可转移性.
科学领域:
- 图表表示学习学习学习图表表示学习.
- 机器学习 机器学习
- 人工智能的人工智能是人工智能.
背景情况:
- 在计算机视觉 (CV) 和自然语言处理 (NLP) 中预训练大型模型可以获得通用知识.
- 图域的进步受到特征和结构异质性的限制.
- 大型语言模型 (LLM) 解决了文本归因图 (TAG) 中的特征异质性,减少了图形结构的重要性.
研究的目的:
- 引入以特征为中心的预训练方法来学习图形表示.
- 利用统一的特征空间来学习通用的交互模式.
- 减轻图形数据中的结构异质性挑战.
主要方法:
- 开发了用变压器 (GSPT) 框架进行图谱序列训练.
- 使用随机走路采样节点上下文.
- 使用变压器在LLM统一的特征空间内使用面具特征重建.
主要成果:
- 通过使用统一文本表示,GSPT减轻了结构异质性.
- 在同一域内的图形中实现了显著更好的可转移性.
- 在节点分类和链接预测任务中表现出有希望的经验成功.
结论:
- GSPT为图形表示学习提供了一个强大的以特征为中心的预训练策略.
- 该框架增强了模型的通用性和可转移性.
- 源代码是公开的可供复制性和进一步研究.
相关概念视频
Transformers
1.7K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.7K
Types Of Transformers
1.4K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.4K
The Ideal Transformer
1.4K
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's tangential...
1.4K
Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
Improving Translational Accuracy
3.6K
3.6K
Transformers with Off-Nominal Turns Ratios
523
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
523
