Hand-Object Pose Estimation Based on Anchor Regression from a Single Egocentric Depth Image.

Area of Science:

• Computer Vision
• Robotics
• Human-Computer Interaction

Background:

• Accurate hand-object pose estimation is crucial for understanding human interaction.
• Severe occlusion presents a significant challenge in vision-based pose estimation tasks.
• Existing methods struggle with precise pose determination in cluttered or occluded scenarios.

Purpose of the Study:

• To develop a robust hand-object pose estimation method capable of handling occlusion.
• To simultaneously estimate the 3D poses of both the hand and the manipulated object.
• To improve the accuracy and reliability of computer vision systems in analyzing human-object interactions.

Main Methods:

• A 3D center detection method was employed to extract foreground hand-object information from depth images.
• An anchor regression approach was utilized within a single framework for simultaneous pose estimation.
• A convolutional neural network (CNN) with ResNet-50 backbone was implemented for keypoint prediction.

Main Results:

• The proposed method achieved mean keypoint errors of 11.85 mm for the hand and 18.97 mm for the object on the FPHA-HO dataset.
• The anchor regression technique effectively estimated poses despite occlusion.
• Accurate pose estimation was demonstrated using single egocentric depth images.

Conclusions:

• The developed anchor regression method provides accurate hand-object pose estimation, even with significant occlusion.
• This advancement enhances the capabilities of computer vision systems for detailed human-object interaction analysis.
• The method shows potential for applications requiring precise understanding of manipulation tasks.