DropNaE: Alleviating irregularity for large-scale graph representation learning.

Area of Science:

• Graph Neural Networks (GNNs)
• GPU computing
• Data science

Background:

• Large-scale graphs are common in real-world applications.
• Graph Neural Networks (GNNs) are effective for processing graph data on GPUs.
• Graph irregularity hinders GPU efficiency during GNN training.

Purpose of the Study:

• To address the inefficiency of GNN training on GPUs caused by graph irregularity.
• To propose a novel method, DropNaE, for mitigating graph data irregularity.
• To enhance both the training speed and accuracy of GNNs.

Main Methods:

• Developed a metric to quantify node neighbor heterophily.
• Introduced DropNaE with two variants to reduce graph irregularity.
• DropNaE preprocesses graphs by conditionally dropping nodes and edges.
• Transformed irregular degree distributions to a more uniform one.

Main Results:

• DropNaE effectively alleviates irregularity in large-scale graphs.
• The method is compatible with popular GNN architectures.
• Experiments demonstrated improvements in both training efficiency and accuracy.
• DropNaE is an offline process requiring no online computational resources.

Conclusions:

• DropNaE offers a significant benefit to current and future state-of-the-art GNNs.
• The proposed method enhances GNN performance by simplifying graph structures.
• DropNaE provides a practical solution for efficient large-scale graph processing.