Geodesic-Aligned Gradient Projection for Continual Task Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks exhibit catastrophic forgetting when trained sequentially.
• Existing gradient projection methods assume static task spaces, limiting continual learning.
• This leads to suboptimal gradient projection and compromised performance on prior tasks.

Purpose of the Study:

• To address catastrophic forgetting in sequential deep learning.
• To develop a method that accounts for gradual task changes.
• To improve continual learning capacity by leveraging geometric properties of task spaces.

Main Methods:

• Embedding task subspaces into a non-Euclidean manifold to capture task evolution.
• Analytically deriving accumulated projection between subspaces along geodesic paths.
• Proposing a novel geodesic-aligned gradient projection (GAGP) method.

Main Results:

• The GAGP method effectively mitigates catastrophic forgetting.
• It utilizes geometric structure information on task manifolds.
• Achieves competitive or superior performance compared to state-of-the-art methods in image classification.

Conclusions:

• The proposed GAGP method offers a robust solution to catastrophic forgetting.
• Non-Euclidean manifolds provide a suitable framework for modeling evolving task spaces.
• This approach enhances the capacity for continual learning in deep networks.