Zero-Shot Image Recognition via Learning Dual Prototype Accordance Across Meta-Domains

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot learning (ZSL) aims to recognize unseen classes by transferring knowledge from seen categories.
• Existing ZSL methods face challenges due to the semantic gap, limited descriptors, and rigid visual modeling.
• Pre-defined class-level attributes as ground truth can hinder semantic-to-visual alignment.

Purpose of the Study:

• To propose a novel ZSL framework, the Bilateral-guided Prototype Refinement Network (BPRN), to address the semantic gap.
• To refine dual prototypes across meta-domains of varying scales for improved ZSL performance.
• To achieve bidirectional calibration between visual-to-semantic and semantic-to-visual modalities.

Main Methods:

• Disentangling class-level semantics to generate pseudo-visual prototypes.
• Leveraging distribution information across dual prototypes in different meta-domains for calibration.
• Employing a synthesized class-level representation from refined dual prototypes for inference.

Main Results:

• BPRN demonstrates competitive or superior performance on five benchmark ZSL datasets.
• Significant improvements observed in the generalized zero-shot learning (GZSL) scenario: 2.1% (AWA1), 7.3% (AWA2), 6.1% (SUN), and 4.8% (aPY).
• Ablation studies and visualization analyses confirm the effectiveness of BPRN's components.

Conclusions:

• The proposed BPRN framework effectively mitigates the semantic gap in ZSL.
• Refining dual prototypes and achieving bidirectional calibration leads to enhanced recognition of unseen classes.
• BPRN offers a promising advancement in embedding-based ZSL methods.