CRNet: A Fast Continual Learning Framework With Random Theory.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Artificial neural networks (ANNs) commonly exhibit catastrophic forgetting, rapidly losing previously learned information when trained on new data.
• This phenomenon is a significant challenge in continual learning, hindering the development of models that can adapt to streaming data.

Purpose of the Study:

• To propose an effective and efficient continual learning framework to address catastrophic forgetting in ANNs.
• To enable a single model to learn from streaming data without compromising performance on previously learned tasks.

Main Methods:

• Developed a continual learning framework integrating random theory and Bayes' rule.
• The core strategy involves guiding output weights to maintain low error regions for old tasks during new task learning.
• The approach offers closed-form solutions, theoretical analysis, and one-pass sample observation for training.

Main Results:

• Demonstrated superior performance over state-of-the-art methods in class incremental learning scenarios.
• Achieved significant advantages in training speed, parameter efficiency, convergence rate, and task-order robustness.
• Experiments on FashionMNIST, CIFAR-100, and ImageNet benchmarks validated the framework's effectiveness.

Conclusions:

• The proposed framework effectively mitigates catastrophic forgetting in artificial neural networks.
• It offers a practical, efficient, and robust solution for continual learning with streaming data.
• The method presents a promising advancement for developing adaptive and continuously learning AI systems.