Few-Shot Human-Object Interaction Recognition With Semantic-Guided Attentive Prototypes Network.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human-Object Interaction (HOI) recognition faces significant challenges due to extreme category imbalance and combinatorial explosion.
• Existing methods often struggle to address both challenges simultaneously.

Purpose of the Study:

• To formulate HOI recognition as a few-shot learning task within a meta-learning framework.
• To propose a novel Semantic-guided Attentive Prototypes Network (SAPNet) to overcome HOI recognition challenges.

Main Methods:

• Developed SAPNet to learn a semantic-guided metric space for HOI recognition.
• Generated attentive prototypes guided by category names to highlight class commonalities.
• Introduced two prototype calculation methods: Prototypes Shift (PS) and Hallucinatory Graph Prototypes (HGP).
• Reorganized benchmark datasets (HICO-NN, TUHOI-NN, HICO-NF, TUHOI-NF) for few-shot HOI tasks.

Main Results:

• SAPNet demonstrated effectiveness in HOI recognition on reorganized benchmark datasets.
• The proposed framework successfully alleviates challenges of data imbalance and combinatorial explosion.
• Attentive prototypes guided by semantic information improved HOI recognition performance.

Conclusions:

• The SAPNet framework offers a robust solution for few-shot HOI recognition.
• The study highlights the potential of meta-learning and semantic-guided prototypes for complex recognition tasks.
• SAPNet provides a promising direction for future research in HOI recognition.