Real-Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Hand gesture recognition offers a flexible and user-friendly human-computer interaction method.
• Developing user-independent systems with high recognition rates is crucial for real-time applications.
• Training deep Convolutional Neural Networks (CNNs) from scratch is challenging due to limited labeled static hand gesture image datasets.

Purpose of the Study:

• To propose an end-to-end fine-tuning method for pre-trained CNN models using score-level fusion.
• To address the challenge of recognizing hand gestures with limited image data.
• To develop and test a real-time American Sign Language (ASL) recognition system.

Main Methods:

• Utilized an end-to-end fine-tuning approach for pre-trained CNN models.
• Implemented a score-level fusion technique to enhance recognition performance.
• Evaluated the method using leave-one-subject-out cross-validation (LOO CV) and regular cross-validation (CV) on benchmark datasets.

Main Results:

• The proposed fine-tuning method with score-level fusion demonstrated effectiveness in recognizing hand gestures from limited image samples.
• The technique was successfully applied to develop and test a real-time American Sign Language recognition system.
• Achieved high recognition performance despite data limitations, indicating the robustness of the approach.

Conclusions:

• The proposed fine-tuning strategy for pre-trained CNNs is a viable solution for hand gesture recognition in low-data scenarios.
• Score-level fusion enhances the performance of CNN models for gesture recognition tasks.
• The developed system shows promise for practical applications like real-time American Sign Language recognition.