A unified gradient regularization method for heterogeneous graph neural networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Heterogeneous Graph Neural Networks (HGNNs) are powerful for learning representations on complex graph data.
• HGNNs suffer from over-smoothing and non-robustness issues.
• Current gradient regularization methods have limitations including training instability and suboptimal heterogeneous information utilization.

Purpose of the Study:

• To propose a novel gradient regularization method, Grug, to address the limitations of existing HGNN techniques.
• To enhance the stability and diversity of HGNNs.
• To provide a unified framework for gradient regularization in HGNNs.

Main Methods:

• Grug iteratively applies regularization to gradients from node type and message matrices during message passing.
• Theoretical analysis was conducted to demonstrate Grug's advantages.
• Extensive experiments were performed on six public datasets.

Main Results:

• Grug demonstrates significant improvements in HGNN performance and effectiveness.
• Theoretical analysis confirms Grug's advantages in stability and diversity.
• Grug potentially surpasses the theoretical upper bounds of DropMessage.

Conclusions:

• Grug offers a robust and effective gradient regularization method for HGNNs.
• The proposed method provides a unified framework and theoretical guidance for optimizing existing techniques.
• Grug significantly enhances the performance and robustness of HGNNs across various tasks and datasets.