Triple-Memory Networks: A Brain-Inspired Method for Continual Learning.

Area of Science:

• Artificial Intelligence
• Neuroscience
• Machine Learning

Background:

• Continual learning in artificial neural networks is hindered by catastrophic forgetting, where new learning overwrites previous knowledge.
• Biological brains overcome this by consolidating memories through region interplay, a process not fully replicated in current AI.
• Existing neural networks struggle to balance learning new tasks with retaining old information.

Purpose of the Study:

• To propose a novel approach, triple-memory networks (TMNs), inspired by brain memory consolidation mechanisms.
• To address the critical challenge of catastrophic forgetting in artificial neural networks.
• To enhance the performance of continual learning systems.

Main Methods:

• Developed a triple-network architecture using generative adversarial networks (GANs) to model brain regions (hippocampus, neocortex, prefrontal cortex).
• Implemented brain-inspired algorithms within each module to manage specific and generalized knowledge representations.
• Utilized a generator for data replay and a weight consolidation regularizer in the discriminator and classifier to prevent information loss.

Main Results:

• TMNs demonstrated state-of-the-art performance in generative memory replay for continual learning.
• Achieved superior results on class-incremental learning benchmarks including MNIST, SVHN, CIFAR-10, and ImageNet-50.
• Effectively mitigated catastrophic forgetting by complementing lost information through a generation process.

Conclusions:

• Triple-memory networks offer a promising brain-inspired solution to catastrophic forgetting in continual learning.
• The proposed architecture effectively balances the acquisition of new knowledge with the retention of previously learned information.
• TMNs represent a significant advancement in developing more robust and adaptable artificial intelligence systems.