A Component-Based Vocabulary-Extensible Sign Language Gesture Recognition Framework.

Area of Science:

• Robotics and Human-Computer Interaction
• Biomedical Engineering
• Artificial Intelligence

Background:

• Sign language recognition (SLR) is crucial for communication accessibility for the deaf.
• Existing SLR systems often require extensive user-specific training data.
• A component-based approach can potentially generalize across users and reduce training burden.

Purpose of the Study:

• To propose and evaluate a component-based, vocabulary-extensible SLR framework.
• To reduce the training data requirements for new users in large-scale gesture recognition.
• To assess the framework's effectiveness using multi-modal sensor data.

Main Methods:

• A component-based framework was developed, decomposing signs into hand shape, axis, orientation, rotation, and trajectory.
• Data from surface electromyographic (sEMG) sensors, accelerometers (ACC), and gyroscopes (GYRO) were utilized.
• A code table was established using a reference subject, and component classifiers were trained for new users with a code matching method.

Main Results:

• The framework achieved high recognition accuracy for 110 Chinese Sign Language (CSL) words, even with small training sets.
• Average recognition accuracy reached (82.6 ± 13.2)% with one-third of the gestures in the training set.
• Accuracy improved to (88 ± 13.7)% with approximately half the gestures in the training set.

Conclusions:

• The proposed component-based SLR framework effectively recognizes large gesture sets with significantly reduced user training.
• This approach lowers the barrier for practical SLR system implementation.
• Multi-modal sensor fusion enhances the robustness and accuracy of sign language recognition.