Related Experiment Video
Updated: Jul 10, 2025
08:25
Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.0K
Point-Based Learnable Query Generator for Human-Object Interaction Detection.
Summary
This study introduces a new Transformer-based framework to improve human-object interaction (HOI) detection. The novel approach enhances feature correlation for more accurate detection of interactions between humans and objects.
Area of Science:
- Computer Vision
- Artificial Intelligence
- Machine Learning
Background:
- Transformer-based and interaction point-based methods show promise in human-object interaction (HOI) detection.
- Directly integrating these distinct model types is challenging due to structural and property differences.
- Current Transformer HOI methods use separate decoders for instance detection and interaction recognition, limiting feature correlation.
Purpose of the Study:
- To propose a novel Transformer-based HOI detection framework that enhances the intrinsic correlation between instance and action features.
- To improve the accuracy of HOI detection by developing a more effective query generation mechanism.
- To advance the state-of-the-art in human-object interaction detection.
Main Methods:
- A novel Transformer-based HOI detection framework is proposed, featuring a decoder with three components: a learnable query generator, an instance decoder, and an interaction classifier.
- The learnable query generator is designed to create effective queries, guiding the instance decoder and interaction classifier to learn accurate instance and interaction features.
- The query generator incorporates prior bounding boxes, keypoint detection, and spatial relation features, inspired by interaction point-based methods.
Main Results:
- The proposed framework demonstrates improved performance in human-object interaction detection.
- Experimental validation on the HICO-DET and V-COCO datasets shows superior results compared to existing state-of-the-art methods.
- The novel learnable query generator effectively enhances the learning of instance and interaction features.
Conclusions:
- The proposed Transformer-based HOI detection framework successfully increases the intrinsic correlation between instance and action features.
- The method achieves better performance on benchmark datasets, indicating its effectiveness and potential for real-world applications.
- The integration of prior bounding boxes, keypoint detection, and spatial relation features in the query generator is a key contribution.
