Synthesizing Depth Hand Images with GANs and Style Transfer for Hand Pose Estimation

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Deep learning for hand pose estimation requires extensive labeled data.
• Generating realistic depth hand images is challenging but crucial for training.
• Existing methods struggle with data scarcity and pose variability.

Purpose of the Study:

• To develop a method for synthesizing depth hand images from ground-truth 3D hand poses.
• To address the data requirement challenge in deep learning for hand pose estimation.
• To improve the performance of hand pose estimation models through data augmentation.

Main Methods:

• Utilized generative adversarial networks (GANs) and image-style transfer techniques.
• Modeled the statistical relationship between 3D hand poses and depth images.
• Ensured synthesized images match training data size and visual appearance.

Main Results:

• Successfully generated depth hand images from 3D hand poses.
• Synthesized images were validated as effective for enhancing model performance.
• Outperformed existing methods on benchmark datasets (NYU, MSRA, ICVL).

Conclusions:

• The developed GAN-based method effectively generates depth hand images.
• Synthesized data significantly improves deep learning models for hand pose estimation.
• This approach offers a viable solution for data augmentation in this field.