Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks.

Area of Science:

• Machine Learning
• Graph Neural Networks
• Artificial Intelligence

Background:

• Graph pre-training (GPT) on graph neural networks (GNNs) learns transferable knowledge from unlabeled data.
• Model success is often linked to large datasets, but this study questions this assumption.
• The 'curse of big data' phenomenon suggests more data doesn't always improve GNN pre-training.

Purpose of the Study:

• To address the 'curse of big data' in GNN pre-training.
• To introduce a 'better-with-less' framework for more efficient graph pre-training.
• To enhance downstream task performance by optimizing data selection.

Main Methods:

• Propose the data-active graph pre-training (APT) framework.
• Utilize a graph selector that identifies representative and instructive data points.
• Incorporate predictive uncertainty from the pre-training model to guide data selection.

Main Results:

• APT framework achieves better downstream performance with fewer training data.
• Demonstrates efficient pre-training by selecting high-quality data points.
• Highlights the effectiveness of predictive uncertainty in guiding the selection process.

Conclusions:

• The APT framework offers an efficient approach to GNN pre-training.
• Careful data selection, guided by predictive uncertainty, overcomes the limitations of large datasets.
• This method leads to improved GNN models for various downstream applications.