SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks.

Area of Science:

• Graph Neural Networks
• Machine Learning
• Deep Learning

Background:

• Graph convolutional networks (GCNs) aggregate neighborhood information to update node features.
• The oversmoothing issue, where node features converge to similar values in deep layers, limits GCN performance.
• Existing methods struggle to effectively address oversmoothing in GCNs.

Purpose of the Study:

• Propose a novel stochastic regularization method to mitigate the oversmoothing problem in GCNs.
• Enhance the performance and generalization capabilities of graph-based models.

Main Methods:

• Introduce Stochastic Feature and Gradient Scaling (SSFG) by applying random scaling factors during training.
• Demonstrate that scaling at both feature and gradient levels is complementary for improved performance.
• Show SSFG can function as a stochastic ReLU activation when combined with ReLU.

Main Results:

• SSFG effectively alleviates the oversmoothing issue by preventing feature convergence.
• The proposed method improves the performance of baseline GCNs across various graph-based tasks.
• Experiments on seven benchmark datasets validate the effectiveness of SSFG regularization.

Conclusions:

• SSFG is an effective regularization technique for GCNs, improving performance without increasing trainable parameters.
• The method offers a simple yet powerful approach to address a key limitation in graph neural networks.
• SSFG enhances the robustness and accuracy of graph-based learning models.