GopGAN: Gradients Orthogonal Projection Generative Adversarial Network With Continual Learning

Area of Science:

• Artificial Intelligence
• Machine Learning

Background:

• Generative adversarial networks (GANs) are powerful for data generation but struggle with catastrophic forgetting in continual learning scenarios.
• Continual learning aims to enable models to learn new tasks sequentially without forgetting previously acquired knowledge.

Purpose of the Study:

• To introduce a novel conditional GAN, GopGAN, designed to mitigate catastrophic forgetting in continual learning.
• To mathematically demonstrate GopGAN's ability to retain knowledge from previously learned tasks.

Main Methods:

• GopGAN updates model weights within an orthogonal subspace defined by training example representations.
• A novel orthogonal projection matrix is derived and an iterative algorithm is proposed for continual learning.
• Task-dependent latent vector construction is employed to maintain the orthogonal subspace for learned tasks.

Main Results:

• GopGAN effectively prevents catastrophic forgetting in generative adversarial networks during continual learning.
• Experiments on diverse datasets (MNIST, EMNIST, SVHN, CIFAR10, ImageNet-200) validate GopGAN's knowledge retention capabilities.
• The proposed method avoids the need to store training data from previous tasks.

Conclusions:

• GopGAN offers a robust solution to catastrophic forgetting in GAN-based continual learning.
• The orthogonal projection approach ensures stable retention of learned knowledge across sequential tasks.
• This work advances continual learning by enabling efficient and effective knowledge preservation in generative models.