Partially Supervised Compositional Zero-Shot Learning by Class-Balanced Distribution Alignment.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Partially supervised Compositional Zero-Shot Learning (pCZSL) faces challenges in recognizing new compositions due to varying state features across objects and scale dependencies.
• Existing methods struggle to effectively model these complex feature interactions.

Purpose of the Study:

• To develop an advanced architecture for pCZSL that accurately recognizes novel object-state compositions.
• To address the variability of state features contingent on object context and scale.

Main Methods:

• A novel architecture employing a Swin transformer-based Hierarchical Feature Extractor (HFE) to capture semantic interactions between state and object features.
• A Discriminative Context Aggregation module to analyze object features at their respective scales using intermediate HFE layers.
• A distribution alignment loss function that minimizes differences between predictions from strongly and weakly augmented images, incorporating class-specific re-weighting to manage data imbalance.

Main Results:

• The proposed method demonstrates superior performance on three benchmark datasets for pCZSL tasks.
• The architecture effectively captures hierarchical features and contextual information crucial for compositional learning.
• The distribution alignment loss with re-weighting successfully leverages partially labeled data and mitigates class imbalance issues.

Conclusions:

• The developed approach significantly advances the state-of-the-art in partially supervised Compositional Zero-Shot Learning.
• The Hierarchical Feature Extractor and Discriminative Context Aggregation modules are effective in handling feature variations and scale dependencies.
• This work provides a robust framework for recognizing complex visual compositions with limited supervision.