Enhanced Scalable Graph Neural Network via Knowledge Distillation.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Graph Representation Learning

Background:

• Graph Neural Networks (GNNs) excel in graph representation learning but face scalability challenges in real-world applications.
• Existing scalable GNNs often compromise effectiveness or offer limited improvements.
• Knowledge Distillation (KD) has proven effective in balancing performance and scalability in other domains.

Purpose of the Study:

• To propose an enhanced scalable GNN via KD (KD-SGNN) that improves both scalability and effectiveness.
• To address the high computational load of GNNs in large-scale graph data scenarios.
• To leverage KD mechanisms for better expressiveness in GNNs.

Main Methods:

• KD-SGNN decouples feature transformation and propagation within GNNs.
• Preprocessing techniques are employed to enhance GNN scalability.
• Two KD mechanisms, soft-target (ST) and shallow imitation (SI) distillation, are introduced to boost expressiveness.

Main Results:

• KD-SGNN demonstrates improved scalability and effectiveness on real-world datasets.
• The proposed KD mechanisms (ST and SI distillation) are verified to enhance GNN expressiveness.
• Comprehensive analyses confirm the benefits of the KD-SGNN approach.

Conclusions:

• KD-SGNN offers a promising solution for scalable and effective GNNs.
• The integration of KD mechanisms significantly enhances GNN performance on large-scale graph data.
• This work contributes to advancing GNN applicability in real-world scenarios.