Learning to Predict Gradients for Semi-Supervised Continual Learning.

Area of Science:

• Machine Intelligence
• Computer Vision
• Machine Learning

Background:

• Continual learning (CL) aims to enable machine intelligence to learn new visual concepts without forgetting prior knowledge.
• Existing CL and semi-supervised CL (SSCL) methods often assume all training samples have known labels, unlike human learning.
• A gap exists between current CL capabilities and human learning, particularly in utilizing unlabeled data.

Purpose of the Study:

• To investigate how unrelated unlabeled data can be utilized in SSCL tasks.
• To understand the impact of unlabeled data on learning and catastrophic forgetting in CL.
• To develop a novel method for integrating unlabeled data into supervised CL frameworks.

Main Methods:

• Formulated a new SSCL method applicable to existing CL models.
• Proposed a novel gradient learner to predict gradients on unlabeled data using labeled data.
• Evaluated the method on mainstream CL, adversarial CL (ACL), and semi-supervised learning (SSL) tasks.

Main Results:

• Achieved state-of-the-art performance in classification accuracy and backward transfer (BWT) within the CL setting.
• Demonstrated desired performance in classification accuracy for SSL tasks.
• Showcased that unlabeled images enhance CL model generalizability and predictive ability for unseen data.

Conclusions:

• Unlabeled data can significantly alleviate catastrophic forgetting in CL models.
• The proposed method effectively integrates unlabeled data into supervised CL, improving performance.
• This approach bridges the gap between machine and human continual learning capabilities.