Layer-Wise Training of Graph Neural Networks With Self-Supervised Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Training graph neural networks (GNNs) on large graphs presents significant memory and computational challenges.
• Scarcity of detailed node-level annotations hinders supervised learning approaches for GNNs.

Purpose of the Study:

• To introduce a scalable self-supervised learning algorithm for training GNNs on large graphs.
• To address the limitations of end-to-end training in terms of memory, computation, and annotation requirements.

Main Methods:

• Propose Layer-wise Regularized Graph Infomax (LRGI), a self-supervised learning algorithm.
• Inspired by predictive coding, LRGI trains GNNs layer by layer, decoupling complexity from network depth.
• Each layer learns to predict neighbor-propagated features, enabling independent training and incorporating regularization for diverse representations.

Main Results:

• LRGI demonstrates scalable training of GNNs on large graphs, significantly improving efficiency.
• The method achieves competitive performance compared to state-of-the-art end-to-end approaches on large inductive graph benchmarks.
• LRGI effectively mitigates the oversmoothing problem in deep GNNs.

Conclusions:

• LRGI offers an efficient and scalable solution for training GNNs on large-scale graph data.
• The layer-wise, self-supervised approach overcomes key limitations of traditional GNN training methods.
• This algorithm facilitates the application of deep GNNs to complex, large-graph problems.