Cognition Guided Human-Object Relationship Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Human-object relationship detection is crucial for comprehensive video understanding.
• Existing methods often overlook specific human information, focusing primarily on object and relation features.
• There is a need for approaches that integrate human-specific cues for more accurate relationship detection.

Purpose of the Study:

• To propose a novel Relation-Pose Transformer (RPT) for human-object relationship detection.
• To incorporate human pose information, specifically head and body pose with skeleton data, into relationship detection.
• To improve the accuracy and comprehensiveness of understanding human-object interactions in videos.

Main Methods:

• Developed a Relation-Pose Transformer (RPT) model.
• Utilized head pose to identify human focus and body pose with skeleton information for action representation.
• Integrated spatial encoding for contextual information and a temporal decoder for temporal dependencies.
• Employed multiple classifiers for diverse relationship predictions.

Main Results:

• The proposed RPT method demonstrates effectiveness on the Action Genome benchmark dataset.
• Achieved state-of-the-art performance compared to existing related methods.
• The integration of pose information significantly enhances human-object relationship detection.

Conclusions:

• The Relation-Pose Transformer (RPT) effectively utilizes human pose information for superior human-object relationship detection.
• The method provides a more comprehensive understanding of videos by analyzing human-object interactions.
• This approach sets a new benchmark in the field of human-object relationship detection.