Zero-Shot Human-Object Interaction Detection via Similarity Propagation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human-object interaction (HOI) detection is crucial for understanding complex scenes.
• Current Transformer-based methods struggle with detecting HOIs involving unseen objects.
• Performance degradation stems from misclassifying novel objects as familiar ones.

Purpose of the Study:

• To improve Transformer-based human-object interaction detection for unseen objects.
• To address the challenge of confusable seen objects misidentified as novel ones.
• To enhance the model's ability to generalize to new interaction categories.

Main Methods:

• Proposed a similarity propagation (SP) scheme using cosine similarity to regulate prediction margins.
• Introduced pseudo-supervision for unseen objects leveraging class semantic similarities.
• Incorporated semantic-aware instance-level and interaction-level contrastive losses with Transformer architecture.
• Enhanced visual representations for better intraclass compactness and interclass separability.

Main Results:

• The proposed SP scheme effectively reduces misclassification of unseen objects.
• Pseudo-supervision aids in learning representations for novel object categories.
• Semantic-aware contrastive losses significantly improve visual feature discrimination.
• The model achieved state-of-the-art performance on V-COCO and HICO-DET benchmarks in zero-shot settings.

Conclusions:

• The developed methods successfully enhance zero-shot HOI detection capabilities.
• The approach mitigates performance degradation caused by unseen objects.
• This work offers a robust framework for generalizing HOI detection to novel scenarios.