HGR-ViT: Hand Gesture Recognition with Vision Transformer.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Hand gesture recognition (HGR) is vital for communication and human-computer interaction.
• Existing deep learning models often neglect crucial hand orientation and positional information.
• There is a need for robust HGR models that capture spatial details.

Purpose of the Study:

• To propose HGR-ViT, a novel Vision Transformer (ViT) model for accurate hand gesture recognition.
• To address limitations in encoding hand position and orientation in previous HGR methods.
• To enhance the performance of HGR systems using an attention-based mechanism.

Main Methods:

• The proposed HGR-ViT model utilizes a Vision Transformer architecture.
• Hand gesture images are divided into fixed-size patches with added positional embeddings.
• A standard Transformer encoder processes these embeddings, followed by a multilayer perceptron for classification.

Main Results:

• HGR-ViT achieved exceptional accuracy: 99.98% on the ASL dataset.
• The model demonstrated high performance on ASL with Digits (99.36%) and NUS datasets (99.85%).
• The inclusion of positional embeddings effectively captures spatial hand information.

Conclusions:

• HGR-ViT significantly advances hand gesture recognition accuracy.
• The model's ability to encode positional information is key to its superior performance.
• This approach offers a promising direction for improved human-computer interaction and sign language recognition.