CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Continual learning (CL) in vision-language models (VLMs) is critical for open-world applications like AI assistants and robotics.
• Existing CL research primarily addresses closed-set scenarios within single domains, overlooking the complexities of diverse, evolving datasets.
• Large pretrained VLMs, such as CLIP, offer strong zero-shot capabilities but struggle with catastrophic forgetting in dynamic environments.

Purpose of the Study:

• To develop a robust open-domain continual learning framework for VLMs.
• To address the challenges of class correlations, domain gaps, and knowledge forgetting in dynamic, multi-domain settings.
• To enhance the adaptability and lifelong learning capabilities of VLMs in real-world applications.

Main Methods:

• Introduced CoLeCLIP, a novel approach for open-domain CL based on the CLIP architecture.
• Employed joint learning of task prompts and a cross-domain class vocabulary to manage diverse data streams.
• Evaluated the method across 11 diverse domain datasets under both task-incremental and class-incremental learning settings.

Main Results:

• CoLeCLIP demonstrated superior performance in open-domain continual learning tasks.
• Achieved new state-of-the-art results in both task-incremental and class-incremental learning settings.
• Successfully mitigated catastrophic forgetting and adapted to novel classes and domains.

Conclusions:

• CoLeCLIP offers a significant advancement in open-domain continual learning for VLMs.
• The proposed method effectively addresses key challenges, including domain shifts and knowledge retention.
• CoLeCLIP paves the way for more capable and adaptable AI systems in open environments.