Neural Prompt Search.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• The exponential growth in vision model size, particularly Vision Transformers, necessitates efficient tuning methods.
• Parameter-efficient tuning methods, like adapter layers and visual prompt tokens, freeze most pre-trained parameters, training only a small subset.
• Designing optimal tuning strategies is complex, requiring extensive experimentation and dataset-specific customization.

Purpose of the Study:

• To introduce Neural prOmpt seArcH (NOAH), a novel approach for automatically designing optimal prompt modules for parameter-efficient tuning of large vision models.
• To adapt prompt module designs dynamically for each specific downstream dataset.
• To enhance the efficiency and effectiveness of fine-tuning large-scale vision architectures.

Main Methods:

• NOAH frames existing parameter-efficient tuning methods as 'prompt modules'.
• It employs a neural architecture search algorithm to discover the optimal prompt module design for each downstream task.
• Extensive experiments were conducted across more than 20 diverse vision datasets.

Main Results:

• NOAH significantly outperforms individual, pre-defined prompt modules.
• The proposed method demonstrates strong performance in few-shot learning scenarios.
• NOAH exhibits robust domain-generalization capabilities across various datasets.

Conclusions:

• NOAH offers a superior and automated solution for parameter-efficient tuning in large vision models.
• The approach effectively addresses the challenges of designing dataset-specific tuning strategies.
• NOAH enhances the adaptability and performance of vision models, particularly in low-data regimes.