3D Hand Pose Estimation Using Synthetic Data and Weakly Labeled RGB Images

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• 3D hand pose estimation from monocular RGB images is difficult due to depth ambiguity and limited annotated data.
• Existing methods often require extensive 3D annotations, which are costly and time-consuming to acquire.
• Leveraging easily obtainable depth images during training offers a potential solution to reduce annotation burden.

Purpose of the Study:

• To develop a weakly-supervised method for 3D hand pose estimation from monocular RGB images.
• To reduce the reliance on fully-annotated 3D data by utilizing depth information during training.
• To improve the accuracy and efficiency of 3D hand pose prediction in real-world scenarios.

Main Methods:

• A weakly-supervised approach adapting from synthetic to real-world RGB datasets using a depth regularizer for weak supervision.
• A novel Conditional Variational Autoencoder (CVAE)-based statistical framework to embed pose-specific subspaces from RGB images.
• Utilizing depth images from RGB-D cameras during training and only RGB input during testing for 3D joint predictions.

Main Results:

• The proposed method effectively alleviates the need for costly 3D annotations in real-world datasets.
• The depth regularizer provides effective weak supervision for 3D pose prediction.
• The CVAE-based framework successfully embeds pose-specific information for accurate 3D joint localization.

Conclusions:

• The proposed weakly-supervised method significantly advances 3D hand pose estimation from monocular RGB images.
• The combination of depth regularization and a CVAE-based framework demonstrates superior performance compared to existing approaches.
• This approach offers a practical solution for real-world applications requiring accurate 3D hand pose estimation with reduced annotation effort.