Toward Generalizable Prompt Learning via Multi-Regularization Guided Knowledge Distillation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Prompt learning in vision-language models (VLMs) enables few-shot/zero-shot learning but suffers from target domain overfitting.
• Overfitting degrades generalization ability and limits performance on unseen categories.

Purpose of the Study:

• To propose a novel multi-regularization guided knowledge distillation method for generalizable prompt learning.
• To enhance the adaptability and generalization of VLMs while mitigating performance degradation during target domain training.

Main Methods:

• Introduced Residual Regularization by adding residual connections to transformer blocks in the image encoder.
• Implemented Self-distillation Regularization to preserve prior generalization knowledge during target domain adaptation.
• Employed unsupervised knowledge distillation with Direction Distillation Regularization for multi-level teacher-student model alignment.

Main Results:

• Demonstrated more stable classification performance in cross-domain few-shot classification.
• Achieved improved performance in domain adaptation settings.
• Showcased enhanced overall model stability and cross-domain adaptability.

Conclusions:

• The proposed multi-regularization approach effectively addresses overfitting in prompt learning for VLMs.
• The method significantly improves generalization and stability for cross-domain tasks.
• This technique offers a promising direction for developing more robust and adaptable vision-language models.