Understanding and mitigating dimensional collapse of Graph Contrastive Learning: A non-maximum removal approach

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Representation Learning

Background:

• Graph Contrastive Learning (GCL) excels at unsupervised graph representation learning (GRL) by maximizing mutual information between graph views.
• A key limitation of GCL is dimensional collapse, where embeddings are confined to a low-dimensional subspace, reducing their expressiveness.

Purpose of the Study:

• To theoretically analyze the causes of dimensional collapse in GCL.
• To propose a novel method, Non-Maximum Removal GCL (nmrGCL), to address dimensional collapse.

Main Methods:

• Theoretical analysis identifying graph pooling and graph convolution regularization as causes of dimensional collapse.
• Development of nmrGCL, which removes prominent dimensions in positive pairs during the contrastive learning pretext task.

Main Results:

• The proposed nmrGCL method effectively mitigates the dimensional collapse problem in GCL.
• Experimental results demonstrate that nmrGCL outperforms existing state-of-the-art methods on multiple benchmark datasets.

Conclusions:

• Dimensional collapse in GCL can be attributed to graph pooling and implicit regularization.
• nmrGCL offers a promising solution to enhance GCL's expressiveness and performance in unsupervised graph representation learning.