Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks.

Area of Science:

• Machine Learning
• Graph Neural Networks
• Network Science

Background:

• Graph neural networks (GNNs) face high computational costs with increasing graph data scale and parameters.
• Existing lottery ticket hypothesis (LTH) methods for GNN sparsification require extensive retraining and ignore node feature redundancy.

Purpose of the Study:

• To develop an efficient graph pruning framework that reduces GNN computational costs.
• To address the limitations of LTH-based methods by avoiding retraining and considering node features.

Main Methods:

• Proposed a comprehensive graph gradual pruning (CGP) framework with during-training pruning.
• Introduced a cosparsifying strategy to prune graph structures, node features, and model parameters.
• Incorporated a regrowth process to re-establish important pruned connections.

Main Results:

• CGP significantly reduces training and inference computation costs compared to LTH methods.
• The framework achieves comparable or superior accuracy across various GNN architectures and datasets.
• Demonstrated effectiveness on node classification tasks, including large-scale Open Graph Benchmark (OGB) datasets.

Conclusions:

• The CGP framework offers an efficient and effective solution for GNN sparsification.
• CGP overcomes limitations of previous methods by integrating dynamic pruning, cosparsification, and regrowth.
• This approach enhances GNN applicability in resource-constrained practical scenarios.