Loss of plasticity in deep continual learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Modern AI relies on artificial neural networks, deep learning, and backpropagation.
• Current methods typically use distinct training and evaluation phases.
• Continual learning, essential for many applications, presents challenges for standard deep learning.

Purpose of the Study:

• To investigate the efficacy of standard deep learning methods in continual learning scenarios.
• To identify the limitations of current deep learning approaches for continuous adaptation.
• To develop and evaluate novel algorithms for maintaining plasticity in deep learning.

Main Methods:

• Testing standard deep learning methods on ImageNet and reinforcement learning tasks.
• Analyzing the loss of plasticity in deep networks during continual learning.
• Introducing and assessing a continual backpropagation algorithm with random unit reinitialization.

Main Results:

• Standard deep learning methods demonstrate a gradual loss of plasticity in continual learning.
• Performance degrades to that of shallow networks without specific interventions.
• The continual backpropagation algorithm successfully maintained plasticity indefinitely.

Conclusions:

• Gradient descent-based methods are insufficient for sustained deep learning in continual settings.
• Injecting diversity, through mechanisms like random reinitialization, is crucial for maintaining plasticity.
• Future deep learning requires hybrid approaches combining gradient-based learning with non-gradient components.