A universal strategy for smoothing deceleration in deep graph neural networks.

Area of Science:

• Machine Learning
• Graph Neural Networks

Background:

• Graph Neural Networks (GNNs) excel at modeling graph data but suffer from over-smoothing in deep layers.
• Existing methods often oversimplify smoothing, neglecting intra-class benefits and neighbor distributions.

Purpose of the Study:

• To propose a novel Smoothing Deceleration (SD) strategy to mitigate over-smoothing in deep GNNs.
• To enhance GNN performance by addressing limitations in current smoothing reduction techniques.

Main Methods:

• Analyzed node representation smoothing speed using differential operations.
• Introduced Class-Related Smoothing Deceleration (CR-SD) loss to balance inter- and intra-class smoothing.
• Developed Smooth Deceleration Residual (NAR) for efficient residual weight computation considering neighbor distributions.

Main Results:

• The SD strategy effectively reduces the smoothing speed rate in deep GNNs.
• CR-SD loss preserves beneficial intra-class smoothing while reducing detrimental inter-class smoothing.
• NAR module improves residual weight calculation by incorporating neighbor node distributions.
• Experimental results show superior performance of SD-enhanced GNNs over baseline and existing deep GNN models.

Conclusions:

• The proposed SD strategy successfully extends shallow GNNs to deeper architectures.
• SD strategy effectively mitigates over-smoothing, leading to improved performance in deep GNNs.
• The novel CR-SD loss and NAR module offer significant advancements in GNN design.