Incremental Learning for Online Data Using QR Factorization on Convolutional Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Neural Networks

Background:

• Catastrophic forgetting is a significant challenge in deep neural networks, leading to rapid loss of learned information when encountering new data.
• Existing incremental learning methods struggle to balance learning new information with retaining previously acquired knowledge.

Purpose of the Study:

• To propose a novel incremental learning framework designed to mitigate catastrophic forgetting in deep neural networks.
• To enable deep neural networks to learn new data and classes in an online manner with reduced knowledge loss.

Main Methods:

• Adoption of hippocampal memory processes to define neural activation boundaries for representing feature distributions.
• Integration of incremental QR factorization to facilitate learning new data with both existing and new labels.
• Development of a framework where feature representations (nodes) are optimized for each class.

Main Results:

• The proposed method effectively alleviates the stability-plasticity dilemma in deep neural networks.
• Experimental results on Cifar-100 and Cifar-10 datasets demonstrate improved performance stability.
• The framework successfully learns unseen data and additional new classes with significantly less forgetting.

Conclusions:

• The novel incremental learning framework successfully addresses catastrophic forgetting in deep neural networks.
• The integration of memory processes and factorization techniques offers a robust solution for continuous learning.
• The method provides a stable yet adaptable learning mechanism for deep neural networks.