Hand Gesture Recognition for Blind Users by Tracking 3D Gesture Trajectory.

Area of Science:

• Human-Computer Interaction
• Assistive Technology
• Machine Learning

Background:

• Hand gestures offer an alternative interaction method for visually impaired individuals.
• Existing gesture recognition algorithms are primarily designed for sighted users and are unsuitable for blind users due to distinct gesture patterns and high inter-user variance.
• Commodity smartwatches can support gesture interaction without specialized sensors.

Purpose of the Study:

• To design an accurate gesture recognition algorithm tailored for blind users.
• To address the challenges posed by the differences and variability in blind users' gestures.
• To enable effective gesture-based interaction for the visually impaired using smartwatches.

Main Methods:

• Developed a gesture recognition algorithm based on a 3D representation of gesture trajectories to capture free-space motion.
• Extracted user-invariant micro-movements within gestures for classification.
• Created an ensemble classifier combining image classification with geometric properties of gestures.

Main Results:

• Achieved a 92% classification accuracy for blind user gestures.
• Demonstrated superior performance compared to the previous state-of-the-art, which had 82% accuracy.
• Successfully identified user-invariant micro-movements for reliable gesture classification.

Conclusions:

• The proposed algorithm effectively recognizes gestures from blind users, overcoming limitations of existing methods.
• The approach leverages unique gesture characteristics and an ensemble classifier for high accuracy.
• This technology has the potential to significantly enhance interaction modalities for visually impaired individuals using smartwatches.