GTC: GNN-Transformer co-contrastive learning for self-supervised heterogeneous graph representation.

Area of Science:

• Graph Representation Learning
• Deep Learning Architectures
• Self-Supervised Learning

Background:

• Graph Neural Networks (GNNs) excel at local information aggregation but suffer from over-smoothing, limiting their depth.
• Transformers offer global information modeling and multi-hop interaction capabilities, showing resilience to over-smoothing.
• Label scarcity in semi-supervised learning restricts the applicability of existing graph methods.

Purpose of the Study:

• To propose a novel framework integrating GNNs and Transformers to address GNN over-smoothing.
• To achieve self-supervised graph representation learning by combining local and global information processing.
• To develop a method that overcomes label scarcity in graph representation learning.

Main Methods:

• Introduced the GTC architecture, a collaborative learning scheme for GNNs and Transformers.
• Utilized separate GNN and Transformer branches for encoding node information from different perspectives.
• Implemented cross-view contrastive learning tasks using encoded information for representation learning.
• Proposed Metapath-aware Hop2Token and CG-Hetphormer for the Transformer branch to encode neighborhood information.

Main Results:

• The GTC framework demonstrated superior performance compared to state-of-the-art methods on real-world datasets.
• The combination of GNN and Transformer effectively mitigated the over-smoothing problem.
• Achieved effective self-supervised heterogeneous graph representation learning.

Conclusions:

• GTC successfully integrates GNN and Transformer capabilities for enhanced graph representation learning.
• The proposed framework offers a robust solution for self-supervised learning in scenarios with limited labeled data.
• This work represents a pioneering effort in leveraging GNN-Transformer collaboration for cross-view contrastive learning in graph representation.