Brave the Wind and the Waves: Discovering Robust and Generalizable Graph Lottery Tickets

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Graph Neural Networks (GNNs) face computational challenges in training and inference on large graphs.
• Existing Graph Lottery Ticket (GLT) methods improve efficiency but struggle with robustness and generalization on diverse, real-world data distributions.

Purpose of the Study:

• To develop a more robust and generalizable approach for GNNs by enhancing the Graph Lottery Ticket methodology.
• To address the limitations of current GNNs in handling out-of-distribution (OOD) data and high graph sparsity.

Main Methods:

• Proposed Resilient Graph Lottery Ticket (RGLT) by reactivating weights/edges using instantaneous gradient information during pruning.
• Implemented environmental interventions to extrapolate potential test distributions.
• Applied model averaging in final pruning stages to enhance generalization.

Main Results:

• RGLT demonstrates improved robustness and generalization capabilities for GNNs.
• The method effectively tackles challenges posed by out-of-distribution data and high graph sparsity.
• Experimental validation across various IID and OOD graph benchmarks confirms RGLT's reliability.

Conclusions:

• RGLT offers a significant advancement in creating more resilient and generalizable GNNs.
• The proposed techniques provide a reliable solution for deploying GNNs in diverse and challenging real-world scenarios.
• This work contributes to overcoming long-standing generalization issues in deep learning for graph data.