A comprehensive study of class incremental learning algorithms for visual tasks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Incremental learning aims for agents to acquire new knowledge without forgetting past data.
• Catastrophic forgetting, where neural networks underfit previous data upon learning new data, is a key challenge.
• Existing research extensively compares methods that increase model capacity but lacks thorough comparison for fixed-size model approaches.

Purpose of the Study:

• To establish a common framework for evaluating fixed-size model approaches in incremental learning.
• To define and analyze desirable properties of incremental learning algorithms.
• To investigate methods for mitigating catastrophic forgetting in a fixed model size.

Main Methods:

• Defined six desirable properties for incremental learning algorithms and analyzed existing methods against them.
• Introduced a unified formalization of the class-incremental learning problem.
• Developed a comprehensive evaluation framework using diverse datasets, sizes, memory constraints, and incremental states.
• Investigated herding for past exemplar selection and assessed performance without knowledge distillation.

Main Results:

• No single fixed-size incremental learning algorithm achieved optimal performance across all evaluated settings.
• Performance varied significantly based on whether a bounded memory of past classes was utilized.
• Competitive performance was demonstrated without relying on knowledge distillation to combat forgetting.

Conclusions:

• The choice of incremental learning algorithm is highly dependent on specific constraints, particularly memory availability.
• A unified framework is crucial for a thorough understanding and comparison of fixed-size incremental learning methods.
• Further research is needed to develop algorithms that robustly handle catastrophic forgetting across diverse incremental learning scenarios.