Zero-Shot Image Classification Method Based on Attention Mechanism and Semantic Information Fusion

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero-shot image classification (ZSIC) addresses classification with limited or missing data by using auxiliary information like attributes or word vectors.
• Existing ZSIC methods struggle with insufficient feature discrimination and limited semantic information, impacting accuracy.
• The mapping between image features and category features is often suboptimal due to these limitations.

Purpose of the Study:

• To improve the accuracy of zero-shot image classification models.
• To enhance the discrimination of image features and the richness of semantic information for unseen classes.
• To overcome the limitations of traditional feature extraction and semantic fusion in ZSIC.

Main Methods:

• A spatial attention mechanism was designed to create an image feature extraction module, focusing on critical and discriminative features.
• A semantic information fusion method utilizing matrix decomposition was proposed to expand information by decomposing attribute features and fusing them with word vector features.
• These methods were integrated to improve the matching degree between image and category features.

Main Results:

• The proposed spatial attention mechanism enhances the extraction of discriminative image features.
• The semantic information fusion method effectively expands and enriches category feature representation.
• Experimental results on public datasets demonstrate significant improvements in ZSIC accuracy for unseen images.

Conclusions:

• The developed spatial attention and semantic fusion techniques effectively enhance zero-shot image classification performance.
• These novel approaches address key limitations in feature extraction and semantic representation for ZSIC.
• The methods show superiority and effectiveness, validated through empirical evaluation on benchmark datasets.