Online continual learning with declarative memory.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks face performance degradation due to catastrophic forgetting when trained sequentially.
• Online multi-task learning presents challenges in retaining knowledge from previous tasks.

Purpose of the Study:

• To propose a novel method, continual learning with declarative memory (CLDM), to mitigate catastrophic forgetting.
• To enhance sequential learning in deep models by leveraging human memory structures.

Main Methods:

• Formulating declarative memory as task memory and instance memory within neural networks.
• Instance memory replays previous samples to retain input-output relations.
• Task memory captures long-term task correlations using a recurrent unit to regularize current task learning.

Main Results:

• CLDM significantly outperforms existing methods on seven continual learning benchmarks.
• The method effectively retains information from both samples and tasks during sequential training.
• Experimental results demonstrate substantial improvements in model performance and knowledge retention.

Conclusions:

• Continual learning with declarative memory (CLDM) offers a powerful solution to catastrophic forgetting.
• The proposed memory system effectively preserves past experiences and facts for sequential learning.
• This approach advances the field of continual learning by enhancing model robustness and adaptability.