Mask-Guided Vision Transformer for Few-Shot Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Few-shot learning (FSL) addresses challenges in training models with limited labeled data.
• Vision Transformers (ViT) are data-intensive models where traditional fine-tuning for FSL can be inefficient.
• Existing FSL methods struggle with knowledge generalization for large-scale models like ViT.

Purpose of the Study:

• To propose a novel mask-guided Vision Transformer (MG-ViT) for effective and efficient few-shot learning.
• To improve the generalization capabilities of ViT models in low-data scenarios.
• To enhance the performance of ViT on downstream tasks like classification, object detection, and segmentation.

Main Methods:

• Introduced a mask-guided Vision Transformer (MG-ViT) that applies masks to image patches.
• Screened task-irrelevant patches to guide the ViT focus on discriminative, task-relevant information.
• Integrated an active learning-based sample selection method for optimal few-shot sample selection.
• Utilized gradient-weighted class activation mapping (Grad-CAM) for mask generation.

Main Results:

• MG-ViT significantly improves performance and efficiency in few-shot learning tasks compared to standard fine-tuning.
• The proposed method demonstrates superior results over general fine-tuning-based ViT and ResNet models.
• MG-ViT effectively generalizes knowledge from pre-trained models without additional computational cost.

Conclusions:

• MG-ViT offers a concrete approach for generalizing data-intensive models like ViT for few-shot learning.
• The mask-guided approach enhances the efficiency and effectiveness of FSL on Vision Transformers.
• This work provides valuable insights for applying large-scale deep learning models in low-data regimes.