Brain-inspired replay for continual learning with artificial neural networks

Area of Science:

• Artificial Intelligence
• Neuroscience
• Machine Learning

Background:

• Artificial neural networks (ANNs) exhibit catastrophic forgetting, rapidly losing previously learned information when trained on new data.
• Human memory relies on neuronal replay mechanisms to consolidate and protect memories, a process not fully replicated in ANNs.
• Current generative replay methods in ANNs struggle with scalability for complex tasks and large datasets.

Purpose of the Study:

• To develop a novel, brain-inspired replay mechanism for artificial neural networks to mitigate catastrophic forgetting.
• To improve the efficiency and scalability of replay techniques in continual learning scenarios.
• To propose a new computational model for memory replay inspired by neural feedback connections.

Main Methods:

• Proposed a new replay method utilizing context-modulated feedback connections to generate and replay internal network representations.
• Implemented a class-incremental learning approach without storing training data.
• Evaluated the method on challenging continual learning benchmarks, including CIFAR-100.

Main Results:

• Achieved state-of-the-art performance on class-incremental learning tasks.
• Demonstrated effective prevention of catastrophic forgetting without the need for data storage.
• Showcased the efficiency and scalability of the proposed brain-inspired replay mechanism.

Conclusions:

• The proposed context-modulated feedback replay offers an efficient and scalable solution to catastrophic forgetting in artificial neural networks.
• This method provides a novel, biologically plausible model for memory replay in neural systems.
• The findings advance the field of continual learning and offer insights into memory consolidation mechanisms.