Learning Domain Invariant Prompt for Vision-Language Models.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Prompt learning adapts large vision-language models (VLMs) efficiently using few samples.
• Current prompt learning struggles with generalization to novel classes and domains.
• Existing methods dynamically generate domain-specific prompts but miss cross-domain generalization potential.

Purpose of the Study:

• Introduce MetaPrompt, a novel paradigm for learning domain-invariant prompts in few-shot scenarios.
• Enhance the generalization capacity of prompts for vision-language models.
• Address limitations in current prompt learning regarding cross-domain and cross-class generalization.

Main Methods:

• Developed a dual-modality prompt tuning network with coupled encoders for independent image and text prompt learning.
• Employed an alternate episodic training algorithm with in-domain and domain-split updates.
• Introduced asymmetric contrastive learning for in-domain updates and domain-split optimization for cross-domain/cross-class tasks.

Main Results:

• MetaPrompt achieved an absolute gain of 1.02% on the overall harmonic mean for base-to-new generalization.
• Demonstrated consistent superiority over benchmarks in domain generalization across 4 datasets.
• Showcased favorable performance across 11 datasets for base-to-new generalization tasks.

Conclusions:

• MetaPrompt effectively learns domain-invariant prompts, significantly improving few-shot generalization.
• The proposed dual-modality network and alternate episodic training enhance model adaptability.
• MetaPrompt offers a promising direction for robust vision-language model adaptation in diverse scenarios.