A Continual Learning Survey: Defying Forgetting in Classification Tasks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Artificial neural networks (ANNs) excel at classification but suffer from catastrophic forgetting when learning new tasks.
• Continual learning (CL) aims to enable ANNs to accumulate knowledge across sequential tasks without retraining.
• Task-incremental classification, where tasks are sequential and distinct, is a key CL paradigm.

Purpose of the Study:

• To provide a taxonomy and overview of state-of-the-art CL methods.
• To introduce a novel framework for assessing the stability-plasticity trade-off in CL.
• To comprehensively compare 11 CL methods and establish baselines.

Main Methods:

• Developed a novel framework to quantify the stability-plasticity trade-off.
• Conducted extensive experimental comparisons of 11 CL methods.
• Evaluated methods on diverse benchmarks including Tiny Imagenet, iNaturalist, and recognition datasets.

Main Results:

• Empirically scrutinized the strengths and weaknesses of various CL methods.
• Analyzed the influence of model capacity, regularization (weight decay, dropout), and task order.
• Qualitatively compared methods based on memory, computation, and storage requirements.

Conclusions:

• The study provides a thorough analysis of the CL landscape for task-incremental classification.
• The proposed framework aids in understanding and optimizing the stability-plasticity balance.
• Empirical results offer insights into method performance and resource utilization for practical CL applications.