Multi-Level Contextual Prototype Modulation for Compositional Zero-Shot Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Compositional Zero-Shot Learning (CZSL) faces challenges due to entangled visual features of attributes and objects, leading to distribution shifts.
• Existing methods often overlook primitive variations and interactions, resulting in poor feature discrimination and biased predictions in CZSL.

Purpose of the Study:

• To propose a novel framework, Multi-level Contextual Prototype Modulation (MCPM), for enhanced performance in Compositional Zero-Shot Learning.
• To address feature entanglement and improve the discriminability of visual embeddings for attribute-object compositions.

Main Methods:

• Developed a transformer-based hierarchical framework (MCPM) to integrate attributes and objects for richer visual embeddings.
• Applied contrastive learning at the feature level for improved discriminability and introduced a subclass-driven modulator for fine-grained attribute-object interactions.
• Implemented a Minority Attribute Enhancement (MAE) strategy to synthesize virtual samples and mitigate data imbalance.

Main Results:

• MCPM demonstrated significant performance improvements across four benchmark datasets (MIT-States, C-GQA, UT-Zappos, VAW-CZSL).
• The proposed methods effectively improved feature discrimination and adaptation to long-tail distributions in compositional tasks.
• The Minority Attribute Enhancement strategy proved effective in mitigating data imbalance issues.

Conclusions:

• The Multi-level Contextual Prototype Modulation (MCPM) framework offers a robust solution for Compositional Zero-Shot Learning.
• MCPM's hierarchical structure and novel strategies significantly enhance the recognition of unseen attribute-object compositions.
• The study validates the effectiveness of MCPM in complex compositional scenes, particularly in handling data imbalance and feature entanglement.