TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers.

Area of Science:

• Computer Science
• Artificial Intelligence
• Graph Analytics

Background:

• Dynamic graphs present challenges due to evolving node features and temporal dynamics.
• Accurate learning requires incorporating long-range historical graph context.
• Existing methods struggle with heterogeneous transient dynamics and varying time intervals.

Purpose of the Study:

• To develop a novel dynamic graph embedding model, TransformerG2G, with uncertainty quantification.
• To effectively learn temporal dynamics by exploiting transformer encoders and historical context.
• To improve performance on tasks like link prediction and node classification.

Main Methods:

• Utilized a transformer encoder to learn intermediate node representations from current and historical graph states.
• Employed projection layers to generate multivariate Gaussian distributions for latent node embeddings.
• Evaluated performance on diverse benchmarks using Temporal Edge Appearance (TEA) plots to measure novelty.

Main Results:

• TransformerG2G outperformed conventional multi-step methods and prior work (DynG2G) in accuracy and efficiency.
• The model demonstrated superior performance, particularly for graphs with a high degree of novelty.
• Learned attention weights revealed automatic adaptive time stepping and identified influential graph elements.

Conclusions:

• TransformerG2G effectively captures temporal dependencies and complex interactions in dynamic graphs.
• The model's attention mechanism provides insights into graph evolution and node influence.
• TransformerG2G offers a robust and efficient solution for dynamic graph analytics.