GTC:GNN-变压器对比学习用于自我监督的异质图表表示
Yundong Sun1, Dongjie Zhu2, Yansong Wang2
1Department of Electronic Science and Technology, Harbin Institute of Technology, Harbin, 150001, China; School of Computer Science and Technology, Harbin Institute of Technology at Weihai, Weihai, 264209, China.
概括
本研究介绍了GTC,这是一个结合图形神经网络 (GNN) 和变压器的新框架,以克服过度平滑并实现自我监督的图形表示学习. 在图表任务中,GTC有效地整合了本地和全球信息,以实现卓越的性能.
科学领域:
- 图形表示学习学习学习图形表示学习
- 深度学习架构 深度学习架构
- 自主监督学习学习
背景情况:
- 图形神经网络 (GNN) 在局部信息聚合方面表现出色,但遭受过度平滑,限制其深度.
- 变压器提供全球信息建模和多跳转交互功能,显示过度平滑的弹性.
- 半监督学习中的标签稀缺性限制了现有的图形方法的适用性.
研究的目的:
- 提出一个整合GNN和变压器的新框架,以解决GNN过度平滑的问题.
- 通过结合本地和全球信息处理来实现自我监督的图形表示学习.
- 开发一种方法来克服图形表示学习中的标签稀缺性.
主要方法:
- 介绍了GTC架构,这是GNN和变压器的协作学习方案.
- 利用单独的GNN和变压器分支来从不同的角度编码节点信息.
- 实施交叉视图对比学习任务,使用编码信息进行表示学习.
- 拟议用于变压器分支的metapath意识Hop2Token和CG-Hetphormer来编码邻里信息.
主要成果:
- 与现实世界数据集上的最先进方法相比,GTC框架表现出更高的性能.
- 结合GNN和变压器有效地缓解了过度平滑的问题.
- 实现了有效的自我监督的异质图表表示学习.
结论:
- GTC成功地集成了GNN和变压器功能,用于增强图形表示学习.
- 拟议的框架为在有限的标记数据的情况下进行自我监督学习提供了强有力的解决方案.
- 这项工作代表了利用GNN-Transformer协作在图形表示中进行交叉视图对比学习的开创性努力.
相关概念视频
Types Of Transformers
951
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
951
Associative Learning
303
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
303
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Improving Translational Accuracy
9.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.3K
Vector Algebra: Graphical Method
11.9K
Vectors can be multiplied by scalars, added to other vectors, or subtracted from other vectors. The vector sum of two (or more) vectors is called the resultant vector or, for short, the resultant.
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
11.9K
Transformers with Off-Nominal Turns Ratios
141
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
141


