Label Deconvolution for Node Representation Learning on Large-Scale Attributed Graphs Against Learning Bias.

Area of Science:

• Graph Neural Networks
• Machine Learning
• Data Science

Background:

• Node representation learning on attributed graphs is vital for downstream tasks.
• Current methods integrate pre-trained models (node encoders) with GNNs.
• Jointly training large models on big graphs presents scalability challenges, leading to separate training and learning bias.

Purpose of the Study:

• To propose an efficient label regularization technique, Label Deconvolution (LD).
• To alleviate learning bias by approximating the inverse mapping of GNNs.
• To incorporate GNN feature convolutions into node encoder training.

Main Methods:

• Developed Label Deconvolution (LD), a scalable label regularization technique.
• Approximated the inverse mapping of GNNs to create an equivalent objective function to joint training.
• Incorporated GNNs into the node encoder training phase to mitigate learning bias.

Main Results:

• LD effectively incorporates GNNs into node encoder training, reducing learning bias.
• LD converges to optimal objective function values achieved by joint training under mild assumptions.
• Experiments on Open Graph Benchmark datasets show LD significantly outperforms state-of-the-art methods.

Conclusions:

• Label Deconvolution (LD) offers an efficient and scalable solution for node representation learning on attributed graphs.
• LD successfully addresses the learning bias issue in separately trained node encoders and GNNs.
• The proposed method demonstrates superior performance compared to existing techniques on benchmark datasets.