Schematic memory persistence and transience for efficient and robust continual learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neuroscience

Background:

• Continual learning in Artificial Intelligence (AI) focuses on enabling deep neural networks (DNNs) to learn sequentially.
• Existing methods primarily address catastrophic forgetting, but reasonable forgetting is crucial for AI performance.
• Human-like learning requires improvements in memory efficiency, generalizability, and robustness to noisy data.

Purpose of the Study:

• To develop a novel framework for continual learning that addresses the limitations of current approaches.
• To enhance AI performance by improving memory efficiency, generalizability, and robustness.
• To introduce a more biologically plausible model for AI continual learning.

Main Methods:

• Proposed the ScheMAtic memory peRsistence and Transience (SMART) framework utilizing external memory.
• Implemented a long-term forgetting mechanism with sparsity and backward positive transfer for efficiency and generalizability.
• Incorporated a short-term forgetting mechanism inspired by background information-gated learning for robustness.

Main Results:

• Demonstrated enhanced memory efficiency and generalizability through theoretical guarantees on error bounds.
• Achieved improved robustness in handling noisy data via the novel short-term forgetting mechanism.
• Validated the effectiveness and efficiency of the SMART framework on benchmark and real-world datasets.

Conclusions:

• The SMART framework offers a promising approach to continual learning by enabling 'reasonable forgetting'.
• The model enhances key aspects of AI performance, bridging the gap towards human-like learning.
• The integration of neuroscience principles provides a foundation for more advanced AI systems.