Semantic-visual shared knowledge graph for zero-shot learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Existing zero-shot learning (ZSL) methods rely on benchmark datasets with pre-defined attributes, limiting their real-world applicability.
• Current approaches using knowledge graphs like WordNet for ZSL face domain shift due to insufficient fine-grained semantics.

Purpose of the Study:

• To propose a Semantic-Visual Shared Knowledge Graph (SVKG) to enhance detailed information for zero-shot learning.
• To address the limitations of existing ZSL methods in real-world scenarios lacking pre-defined attributes.

Main Methods:

• Developed SVKG integrating semantic embeddings for high-level information and visual features for fine-grained descriptions.
• Proposed a multi-modal graph convolution network to transform SVKG into graph representations for ZSL tasks.
• Utilized visual features directly extracted from real-world images to substitute pre-defined attributes.

Main Results:

• Demonstrated effectiveness on real-world datasets without pre-defined attributes.
• Achieved performance improvements of +2.8% (2-hops), +0.5% (3-hops), and +0.2% (all divisions) over state-of-the-art methods.
• Validated the benefits of integrating visual features for fine-grained ZSL.

Conclusions:

• SVKG effectively enhances zero-shot learning by providing detailed fine-grained information.
• The proposed method offers a viable solution for applying ZSL to real-world datasets.
• The integration of visual features within a knowledge graph framework is crucial for overcoming domain shift in ZSL.