FingerPoseNet: A finger-level multitask learning network with residual feature sharing for 3D hand pose estimation

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Current 3D hand pose estimation methods often use shared feature maps, limiting the enhancement of crucial finger-level details.
• Accurate joint-to-finger associations and articulations are vital for precise hand pose estimation but are challenging to capture with existing techniques.

Purpose of the Study:

• To introduce FingerPoseNet, a novel finger-level multitask learning network for accurate 3D hand pose estimation from depth images.
• To address the limitations of current methods in enhancing finger-level features for improved hand articulation analysis.

Main Methods:

• FingerPoseNet utilizes a three-stage architecture: a ResNet-50 backbone for shared feature extraction, a finger-level multitask learning stage for enhancing individual finger and palm features, and a multitask fusion layer.
• Employs multitask learning by decomposing hand pose estimation into six subtasks (one for each finger and the palm), each handling feature extraction, enhancement, and 3D keypoint regression.
• Introduces a residual feature-sharing approach to mine supplementary information across all subtasks, enhancing subtask-specific features.

Main Results:

• FingerPoseNet demonstrates significant improvements in accuracy compared to state-of-the-art approaches.
• Experiments conducted on five challenging public datasets (ICVL, NYU, MSRA, Hands-2019-Task1, HO3D-v3) validate the effectiveness of the proposed method.

Conclusions:

• FingerPoseNet effectively addresses the challenge of enhancing finger-level features in 3D hand pose estimation.
• The proposed finger-level multitask learning network with residual feature sharing offers a robust and accurate solution for estimating 3D hand poses from depth data.