Few-shot human-object interaction video recognition with transformers.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Recognizing human-object interactions (HOI) is crucial for understanding complex actions in videos.
• Few-shot learning presents a significant challenge due to the scarcity of labeled data for many HOI classes.

Purpose of the Study:

• To develop a novel few-shot learning framework for accurate human-object interaction recognition.
• To leverage meta-learning and transformer architectures to embed HOI into compact features for efficient similarity calculation.

Main Methods:

• A meta-learning paradigm is employed to learn from limited labeled samples.
• Transformer-based spatial and temporal encoders are utilized to capture relationships within HOI.
• A spatial encoder extracts frame-level features, and a temporal encoder generates video-level representations.

Main Results:

• The proposed framework achieved significant accuracy improvements on the CAD-120 and Something-Else datasets.
• Specifically, 7.8% and 15.2% accuracy gains were observed for the 1-shot task.
• Improvements of 4.7% and 15.7% were recorded for the 5-shot task, outperforming existing state-of-the-art methods.

Conclusions:

• The novel few-shot learning framework effectively recognizes human-object interactions with limited data.
• The integration of meta-learning and transformers provides a robust approach for HOI recognition.
• The method demonstrates superior performance compared to current state-of-the-art techniques.