Repeated Cross-Scale Structure-Induced Feature Fusion Network for 2D Hand Pose Estimation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Hand pose estimation from RGB images has advanced with convolutional neural networks.
• Inferring self-occluded keypoints remains a significant challenge in hand pose estimation.
• Traditional appearance features are insufficient for recognizing occluded keypoints, necessitating contextual information.

Purpose of the Study:

• To address the challenge of self-occluded keypoint inference in hand pose estimation.
• To propose a novel network architecture that enhances feature learning using contextual information.
• To improve the accuracy of 2D hand pose estimation, particularly for occluded keypoints.

Main Methods:

• A repeated cross-scale structure-induced feature fusion network is proposed.
• The network comprises two modules: GlobalNet for initial joint localization and RegionalNet for refinement.
• GlobalNet utilizes a feature pyramid structure combining semantic and global spatial information.
• RegionalNet employs a four-stage cross-scale feature fusion to learn appearance features informed by hand structure.

Main Results:

• The proposed method demonstrates superior performance compared to state-of-the-art techniques.
• Evaluated on the STB and RHD public datasets for 2D hand pose estimation.
• Achieved improved accuracy in locating keypoints, especially under self-occlusion conditions.

Conclusions:

• The novel network effectively learns keypoint representations by integrating multi-level features and contextual relationships.
• The proposed approach significantly enhances the capability to infer self-occluded keypoints in hand pose estimation.
• The method represents a substantial advancement in 2D hand pose estimation accuracy and robustness.