Effects of Motion-Relevant Knowledge From Unlabeled Video to Human-Object Interaction Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human-object interaction (HOI) detection models struggle with insufficient labeled data, particularly for rare interaction categories.
• Existing methods require large labeled datasets or rely heavily on pre-learned knowledge, limiting real-world applicability.

Purpose of the Study:

• To propose a multitask learning (MTL) framework that utilizes unlabeled videos to enhance HOI detection.
• To address the challenge of limited labeled data for rare HOI categories by incorporating motion-relevant information.

Main Methods:

• A multitask learning (MTL) approach was developed, integrating self-supervised learning on unlabeled videos.
• Key components include appearance reconstruction loss (ARL) and a sequential motion mining module to learn generalizable motion representations.
• A domain discriminator was employed to bridge the domain gap between unlabeled videos and HOI images.

Main Results:

• The proposed method demonstrated effectiveness in improving HOI detection, particularly for rare categories on the HICO-DET dataset.
• Experiments on the V-COCO dataset showed strong performance under minimum supervision conditions.
• The study confirmed the value of motion-aware knowledge from unlabeled videos for HOI detection.

Conclusions:

• Leveraging unlabeled videos through multitask learning and self-supervised motion representation learning is a viable strategy for enhancing HOI detection.
• This approach effectively mitigates the limitations of scarce labeled data, especially for rare HOI categories.
• The findings highlight the potential of utilizing readily available unlabeled video data in computer vision tasks.