A Dual-Branch Self-Boosting Framework for Self-Supervised 3D Hand Pose Estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Deep neural networks have advanced 3D hand pose estimation.
• Labeled data collection for these methods is labor-intensive and time-consuming.
• Self-supervised learning offers a promising alternative to reduce data dependency.

Purpose of the Study:

• To develop a self-supervised framework for accurate 3D hand pose estimation from depth images.
• To overcome the limitations of data-hungry supervised methods.
• To improve the efficiency and robustness of hand pose estimation.

Main Methods:

• A dual-branch self-boosting framework utilizing image-to-image translation for synthetic data pre-training.
• Decoupled 3D hand model and pixel-wise pose estimation.
• Part-aware model-fitting loss and inter-branch loss for continuous self-supervised learning.
• A refinement stage leveraging estimated hand model structure.

Main Results:

• Outperformed previous self-supervised methods significantly without multi-view data.
• Achieved comparable results to strongly supervised methods.
• Demonstrated substantial improvement in skeleton-based gesture recognition using regenerated pose annotations.

Conclusions:

• The proposed dual-branch self-boosting framework enables effective self-supervised 3D hand pose estimation from depth images.
• The method significantly reduces reliance on labeled data while achieving high accuracy.
• This approach offers a viable solution for real-world applications requiring efficient and robust hand pose tracking.