Continual Learning by Contrastive Learning of Regularized Classes in Multivariate Gaussian Distributions.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks suffer from catastrophic forgetting during incremental updates, hindering continuous learning.
• Existing continual learning (CL) methods struggle to maintain past knowledge due to overlapping stored information.

Purpose of the Study:

• To propose a novel CL method that effectively preserves previously learned knowledge.
• To enhance model plasticity and discriminability for improved incremental task performance.

Main Methods:

• Preserving knowledge as multivariate Gaussian distributions by storing and reproducing model outputs per class.
• Utilizing contrastive learning and representation regularization to improve class separation and adaptability.
• Storing class-wise spatial means and covariances in latent space for knowledge retention.

Main Results:

• Achieved high accuracies on CIFAR-10 (93.21%), CIFAR-100 (77.57%), and ImageNet-100 (78.15%).
• Outperformed state-of-the-art CL methods by significant margins (2.34%p, 2.1%p, 1.91%p).
• Demonstrated the lowest mean forgetting rates across all tested benchmark datasets.

Conclusions:

• The proposed method effectively preserves previous knowledge in incremental learning scenarios.
• The approach offers a promising solution to catastrophic forgetting in deep neural networks.
• The method enhances both knowledge retention and adaptability for sequential tasks.