Attribute-driven streaming edge partitioning with reconciliations for distributed graph neural network training.

Area of Science:

• Graph Neural Networks
• Distributed Systems
• Machine Learning

Background:

• Current distributed graph training frameworks suffer from high communication costs and slow convergence due to separate storage and training designs.
• Traditional graph partitioning methods incur memory overhead and damage semantic structures by ignoring node attributes.
• Heterogeneous local models in distributed training hinder convergence through simple averaging synchronization.

Purpose of the Study:

• To propose a novel distributed graph training approach, attribute-driven streaming edge partitioning with reconciliations (ASEPR).
• To reduce communication costs and memory overhead in distributed graph training.
• To improve convergence speed and global model performance with heterogeneous local models.

Main Methods:

• ASEPR clusters nodes with similar attributes to maintain semantic structure and neighbor locality.
• Streaming partitioning combined with attribute clustering is used for efficient subgraph assignment, alleviating memory overhead.
• Cross-layer reconciliation strategies, including knowledge distillation and contrastive learning, are employed to enhance the global model from heterogeneous local models.

Main Results:

• ASEPR outperforms existing methods like DistDGL on node classification and link prediction tasks.
• The proposed approach requires fewer computational resources.
• ASEPR achieves up to a quadruple increase in convergence speed compared to baseline methods.

Conclusions:

• ASEPR offers an effective solution for distributed graph training by optimizing graph partitioning and model reconciliation.
• The method successfully addresses the limitations of traditional frameworks, leading to improved efficiency and performance.
• ASEPR demonstrates significant advantages in terms of resource utilization and training speed for large-scale graph analysis.