American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach

Area of Science:

• Computer Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Sign language facilitates communication for deaf and dumb communities.
• Limited societal adoption of sign language learning necessitates technological solutions.
• Existing sign language recognition systems often focus on incomplete sets of gestures.

Purpose of the Study:

• To develop a sign language recognition prototype for full American Sign Language (ASL) recognition, including 26 letters and 10 digits.
• To differentiate between static and dynamic ASL gestures by extracting features from finger and hand motions.
• To evaluate the performance of Support Vector Machine (SVM) and Deep Neural Network (DNN) models for ASL recognition.

Main Methods:

• Utilized the Leap Motion Controller (LMC) for capturing hand and finger movements.
• Implemented feature extraction techniques to analyze both static and dynamic gestures.
• Trained and evaluated Support Vector Machine (SVM) and Deep Neural Network (DNN) models.

Main Results:

• Deep Neural Network (DNN) achieved a recognition rate of 93.81% for 26 ASL letters.
• Support Vector Machine (SVM) achieved a recognition rate of 80.30% for 26 ASL letters.
• Recognition rates for the combined set of 26 letters and 10 digits were 88.79% (DNN) and 72.79% (SVM).

Conclusions:

• The developed sign language recognition system demonstrates significant potential for improving communication accessibility.
• The prototype can serve as a valuable interpreter for deaf and dumb individuals in daily service interactions.
• The study highlights the effectiveness of DNNs in recognizing complex ASL gestures.