Exploiting domain transformation and deep learning for hand gesture recognition using a low-cost dataglove

Area of Science:

• Human-Computer Interaction
• Deep Learning
• Wearable Technology

Background:

• Hand gesture recognition is a key area in human-computer interaction, experiencing renewed interest due to advancements in hardware and deep learning.
• Traditional methods often require expensive equipment or complex setups.
• The need for cost-effective and accurate solutions for recognizing both static and dynamic gestures persists.

Purpose of the Study:

• To evaluate the effectiveness of a low-cost dataglove for classifying hand gestures using deep learning.
• To develop and validate a generalized hand gesture recognition system.
• To introduce novel techniques for dynamic gesture recognition and multimodal data processing.

Main Methods:

• Development of a cost-effective dataglove incorporating flex sensors, an inertial measurement unit, and a microcontroller for onboard processing and wireless connectivity.
• Data collection from 25 subjects performing 24 static and 16 dynamic American Sign Language gestures.
• Proposal of a novel Spatial Projection Image-based technique for dynamic gesture recognition and exploration of a parallel-path neural network for multimodal data.

Main Results:

• The system achieved an F1-score of 82.19% for static gestures and 97.35% for dynamic gestures.
• Validation was performed using a leave-one-out cross-validation approach.
• The developed system demonstrated promising performance for generalized hand gesture recognition.

Conclusions:

• The low-cost dataglove, combined with deep learning techniques, offers a highly effective solution for hand gesture recognition.
• The proposed methods show significant potential for real-world applications in human-computer interaction.
• The publicly released dataset will facilitate further research and development in the field.