Modality independent adversarial network for generalized zero shot image classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Zero Shot Learning (ZSL) aims to classify images of unseen classes by transferring knowledge from seen classes via semantic embeddings.
• Current ZSL methods struggle to fully capture cross-modal semantic consistency, resulting in less discriminative representations.

Purpose of the Study:

• To propose a novel deep framework, Modality Independent Adversarial Network (MIANet), for Generalized Zero Shot Learning (GZSL).
• To address limitations in existing ZSL approaches regarding cross-modal semantic consistency and representation discriminability.

Main Methods:

• Embedding visual features and semantic descriptions into a joint latent hyper-spherical space with orthogonal constraints for discriminative representations.
• Utilizing a modality adversarial submodule to ensure latent representations are modality-independent, capturing more cross-modal semantic information.
• Employing a cross reconstruction submodule to reconstruct latent representations into counterparts, enhancing modality-irrelevant information capture.

Main Results:

• Comprehensive experiments on five benchmark datasets demonstrate the effectiveness of MIANet in both GZSL and standard ZSL settings.
• The proposed orthogonal constraints, modality adversarial, and cross reconstruction submodules contribute to improved performance.

Conclusions:

• MIANet offers an effective end-to-end deep architecture for Generalized Zero Shot Learning.
• The framework successfully improves the discriminability and semantic consistency of cross-modal latent representations.