Multi-Label Zero-Shot Learning Via Contrastive Label-Based Attention

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Multi-label zero-shot learning (ML-ZSL) aims to identify all objects in an image, including those not seen during training.
• Current ML-ZSL methods use attention mechanisms but suffer from semantic ambiguity due to equal treatment of label embeddings.
• This ambiguity hinders accurate object recognition when multiple labels are present.

Purpose of the Study:

• To enhance the utilization of semantic information within attention mechanisms for ML-ZSL.
• To propose a novel method that reduces semantic ambiguity in label prediction.
• To improve the accuracy of recognizing unseen object categories in images.

Main Methods:

• Introduction of a contrastive label-based attention (CLA) method.
• CLA associates each label with the most relevant image regions using latent label embeddings.
• Implementation of a region-level contrastive loss and a global feature alignment module.

Main Results:

• CLA effectively captures discriminative image details and distinguishes region-wise correlations.
• Experiments on NUS-WIDE and Open Images benchmarks show CLA outperforms state-of-the-art methods.
• Significant improvements in mean Average Precision (mAP): 2.0% on NUS-WIDE and 4.0% on Open Images under the zero-shot learning setting.

Conclusions:

• The proposed CLA method significantly reduces semantic ambiguity in ML-ZSL.
• CLA demonstrates superior performance in identifying unseen object categories.
• The approach offers a more efficient and accurate way to leverage semantic information in attention mechanisms.