A graph transformer with optimized attention scores for node classification.

Area of Science:

• Graph Neural Networks
• Transformer Architectures
• Machine Learning

Background:

• Message-passing on graphs excels at local structures but struggles with global information.
• Transformer models, while strong in nonlocal modeling, underperform GNNs in node-level prediction tasks.
• Existing research focused on approximating Transformers, neglecting their potential for node embedding.

Purpose of the Study:

• Introduce OGFormer, a novel graph Transformer model with optimized attention scores.
• Address limitations in capturing global dependencies and complex relationships in graph data.
• Enhance node embedding representation learning for improved graph analysis.

Main Methods:

• Developed OGFormer, a graph Transformer with a simplified single-head self-attention mechanism.
• Implemented an end-to-end attention score optimization loss function to refine connection weights.
• Integrated a structural encoding strategy into attention computation to prioritize key local dependencies.

Main Results:

• OGFormer demonstrates competitive performance in node classification tasks across benchmark datasets.
• Achieved superior results in both Homophilous and Heterophilous graph tests.
• Outperformed current mainstream graph neural network (GNN) methods.

Conclusions:

• OGFormer effectively captures global dependencies and local structures in graphs.
• The proposed attention optimization and structural encoding enhance node embedding representation.
• OGFormer represents a significant advancement for node classification using Transformer-based graph models.