Distributed Optimization of Graph Convolutional Network Using Subgraph Variance.

Area of Science:

• Machine Learning
• Graph Neural Networks
• Distributed Systems

Background:

• Distributed Graph Convolutional Networks (GCNs) excel at learning representations from large graph-structured data.
• Existing frameworks suffer from high communication costs due to extensive data transmission between processors.

Purpose of the Study:

• To propose a novel distributed GCN framework (GAD) that minimizes communication overhead.
• To enhance the efficiency and accuracy of distributed GCN training on large-scale graphs.

Main Methods:

• Developed GAD-Partition for augmentation-based graph partitioning to reduce inter-processor communication.
• Introduced GAD-Optimizer with subgraph variance-based importance calculation and weighted global consensus.
• Designed to adaptively adjust subgraph importance and mitigate variance introduced by partitioning.

Main Results:

• Achieved significant reduction in communication overhead (approximately 50%).
• Improved convergence speed of distributed GCN training by approximately 2x.
• Obtained a slight accuracy gain (approximately 0.45%) with minimal redundancy.

Conclusions:

• The proposed GAD framework effectively reduces communication costs in distributed GCN training.
• GAD enhances convergence speed and accuracy for large-scale graph representation learning.
• This approach offers a promising solution for efficient distributed GCN training.