Catastrophic Forgetting in Deep Graph Networks: A Graph Classification Benchmark.

Area of Science:

• Machine Learning
• Graph Representation Learning
• Continual Learning

Background:

• Catastrophic forgetting is a key challenge in continual learning, where models forget previously learned information when trained on new data.
• Applying continual learning techniques to graph representation learning is underexplored, despite the prevalence of graph-structured data.

Purpose of the Study:

• To investigate the effectiveness of classical continual learning techniques on graph representation learning tasks.
• To evaluate the impact of structure-preserving regularization on mitigating catastrophic forgetting in graph neural networks.

Main Methods:

• Experiments were conducted using a structure-agnostic model and a deep graph network on three distinct datasets.
• The study benchmarked the performance of different continual learning strategies, including replay, and assessed the role of regularization.

Main Results:

• Replay emerged as the most effective strategy for combating catastrophic forgetting in the tested graph scenarios.
• The benefits of replay were significantly enhanced when combined with structure-preserving regularization techniques.

Conclusions:

• Classical continual learning methods, particularly replay, show promise for graph representation learning.
• Further research is warranted at the intersection of continual and graph representation learning, with regularization being a key component.