Three types of incremental learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Neural Networks

Background:

• Continual learning enables systems to learn from non-stationary data streams, mimicking natural intelligence.
• Deep neural networks struggle with continual learning, despite numerous proposed methods.
• Comparing continual learning strategies is challenging due to the absence of a standardized framework.

Purpose of the Study:

• To introduce a categorization of continual learning into three fundamental scenarios: task-incremental, domain-incremental, and class-incremental learning.
• To provide a comprehensive empirical comparison of existing continual learning strategies across these scenarios.
• To establish a foundation for defining benchmark problems and structuring the continual learning field.

Main Methods:

• Categorization of continual learning into task-incremental, domain-incremental, and class-incremental scenarios.
• Empirical comparison of current continual learning strategies.
• Utilizing Split MNIST and Split CIFAR-100 datasets and protocols.

Main Results:

• Significant differences in difficulty and strategy effectiveness were observed across the three continual learning scenarios.
• The proposed categorization highlights scenario-specific challenges.
• Performance of different strategies varies considerably depending on the continual learning scenario.

Conclusions:

• The three-scenario categorization provides a structured approach to understanding and evaluating continual learning methods.
• This framework facilitates clearer definition of benchmark problems for deep neural networks.
• Further research can build upon this categorization to advance the field of continual learning.