Learning to Model Relationships for Zero-Shot Video Classification.

Area of Science:

• Machine Learning
• Pattern Analysis
• Computer Vision

Background:

• Zero-shot learning (ZSL) is crucial for video classification due to the expanding number of categories.
• Existing ZSL methods struggle with effective knowledge transfer between seen and unseen classes.
• This is often due to inadequate modeling of relationships between categories and attributes.

Purpose of the Study:

• To develop a more effective zero-shot learning framework for video classification.
• To address the limitations of existing methods in modeling concept relationships.
• To improve knowledge transfer from seen to unseen video categories.

Main Methods:

• Proposed a novel prototype-sample graph neural network (PS-GNN) for video ZSL.
• PS-GNN models relationships between category-attribute, category-category, and attribute-attribute explicitly.
• Employs a task-driven message passing process with a prototype branch and a sample branch for joint learning.

Main Results:

• PS-GNN demonstrated consistent and favorable performance across five benchmark datasets.
• The method effectively alleviates the heterogeneity gap and domain shift.
• Robust temporal modeling capabilities were achieved.

Conclusions:

• The proposed PS-GNN offers a unified and robust framework for video zero-shot learning.
• Explicitly modeling concept relationships enhances knowledge transfer effectiveness.
• PS-GNN represents a significant advancement in machine learning for video pattern analysis.