Semi-Supervised Joint Learning for Hand Gesture Recognition from a Single Color Image

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Hand gesture recognition and hand pose estimation are related tasks.
• Existing methods may lack robustness in unconstrained environments.
• Data annotation for these tasks can be challenging.

Purpose of the Study:

• To develop a deep learning approach for joint hand gesture recognition and 3D hand pose estimation.
• To improve gesture recognition accuracy by leveraging shared features.
• To address data scarcity using a semi-supervised training scheme.

Main Methods:

• A deep learning model that jointly learns shared features for both tasks.
• A semi-supervised training strategy to handle limited annotations.
• Simultaneous foreground hand detection, gesture recognition, and 3D pose estimation.

Main Results:

• The proposed approach achieves significantly improved gesture recognition accuracy.
• Leveraging shared features from hand pose estimation enhances gesture recognition.
• A new dataset for evaluating hand gesture recognition in unconstrained settings was introduced.

Conclusions:

• Jointly learning features for correlated tasks is an effective strategy.
• Semi-supervised learning is beneficial for training models with limited data.
• The proposed method offers a robust solution for hand gesture recognition and pose estimation.