Multiple semantic prompt for rehearsal-free continual learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks face catastrophic forgetting, losing performance on old tasks when learning new ones.
• Continual learning seeks to mitigate this issue, with prompt-based methods emerging as a promising solution without data replay.
• Existing prompt methods may not fully leverage semantic information for robust continual learning.

Purpose of the Study:

• To develop a novel prompt-based continual learning method that enhances semantic information retention.
• To improve zero-shot transfer capabilities in vision-language models within a continual learning setting.
• To overcome limitations of traditional data replay and basic prompt methods.

Main Methods:

• Proposed Multiple Semantic Prompt (MSP) method linking prompts and textual labels to capture semantic information.
• Combined prompts based on the similarity between visual and textual features.
• Utilized pre-trained models for zero-shot transfer instructed by the combined prompts.

Main Results:

• MSP demonstrated superior performance over state-of-the-art prompt-based methods, achieving over 3% higher average final accuracy.
• Achieved over 15% higher zero-shot classification accuracy compared to vanilla vision-language models on benchmarks.
• Effectively retained knowledge across tasks without requiring a rehearsal buffer.

Conclusions:

• The Multiple Semantic Prompt (MSP) method is a highly effective approach for continual learning, significantly mitigating catastrophic forgetting.
• MSP enhances semantic understanding and improves zero-shot transfer capabilities, offering a privacy-preserving alternative to data replay.
• The proposed method represents a significant advancement in prompt-based continual learning strategies.