Continuous learning in single-incremental-task scenarios.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Sequential training of deep models can suffer from catastrophic forgetting.
• Existing strategies (architectural, regularization, rehearsal) are effective for disjoint tasks but not single incremental tasks.
• Class-incremental learning presents unique challenges not adequately addressed by current methods.

Purpose of the Study:

• Differentiate multi-task and single-incremental-task learning scenarios.
• Evaluate limitations of existing methods like LWF, EWC, and SI in incremental learning.
• Propose a novel, efficient approach for class-incremental learning.

Main Methods:

• Introduced AR1, a novel approach combining architectural and regularization strategies.
• Focused on minimizing memory and computational overhead for online learning suitability.
• Evaluated AR1 on benchmark datasets CORe50 and iCIFAR-100.

Main Results:

• Demonstrated that AR1 significantly outperforms existing regularization strategies.
• Showcased AR1's effectiveness in class-incremental learning settings.
• Confirmed AR1's low overhead, making it suitable for online learning.

Conclusions:

• AR1 offers a promising solution for the challenges of class-incremental learning.
• The proposed method effectively mitigates catastrophic forgetting in sequential learning.
• AR1 provides a computationally efficient and high-performing alternative for incremental deep learning.