Multiscale Visual-Attribute Co-Attention for Zero-Shot Image Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot image recognition (ZSR) aims to classify data from unseen classes by linking visual features with semantic representations.
• Current ZSR methods often use a single-scale embedding space, overlooking the varying semantics present in different visual feature scales.

Purpose of the Study:

• To propose a novel multi-scale visual-attribute co-attention (mVACA) model for enhanced zero-shot image recognition.
• To address limitations of single-scale approaches by incorporating multi-scale visual semantics and improving visual discrimination.

Main Methods:

• The mVACA model employs a hybrid visual attention mechanism at each scale, combining attribute-related attention and visual self-attention.
• Attribute-related attention is guided by pseudo-attribute vectors derived from mutual information regularization (MIR).
• Visual self-attention refines attribute attention, emphasizing visually relevant attributes and leveraging multi-scale visual discrimination.

Main Results:

• The mVACA model demonstrates state-of-the-art or competitive performance on standard benchmarks for both zero-shot learning (ZSL) and generalized ZSL (GZSL) tasks.
• The framework effectively unifies standard ZSL and GZSL by utilizing multi-scale visual discrimination.

Conclusions:

• The proposed mVACA model offers a significant advancement in zero-shot image recognition by effectively integrating multi-scale visual and semantic information.
• The model's ability to handle both ZSL and GZSL tasks, coupled with visualized analysis of image-attribute interactions, provides valuable insights into ZSR mechanisms.