Attribute Prompt Alignment Network for Zero-Shot Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Traditional zero-shot learning (ZSL) relies on category attributes for knowledge transfer.
• Large language models like CLIP use category names for ZSL-like predictions.
• Existing methods struggle to integrate CLIP's strengths into traditional ZSL frameworks.

Purpose of the Study:

• To improve knowledge transferability from pretrained CLIP models to downstream ZSL frameworks.
• To develop generalizable feature representations by leveraging attribute information.
• To investigate the impact of CLIP on ZSL in the era of large models.

Main Methods:

• Attribute Prompt Tuning (APT) to generate attribute prompts from class descriptions.
• Cross-Network Feature Alignment (CFA) using a two-branch APAN architecture.
• Visual-semantic interaction attention to guide visual region localization across frozen networks.
• Prediction alignment loss to constrain predictions from cross-network visual features.

Main Results:

• APAN progressively refines and aligns cross-network features.
• The approach captures fine-grained attribute information for generalizable representations.
• APAN outperforms state-of-the-art methods on three benchmark datasets.
• APAN effectively absorbs generalizable knowledge from CLIP models.

Conclusions:

• APAN successfully bridges the gap between CLIP and traditional ZSL.
• The proposed method enhances feature representations for improved ZSL performance.
• APAN offers a viable approach for leveraging large pretrained models in ZSL tasks.