A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface...

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Sign language recognition (SLR) is crucial for deaf communication.
• Existing SLR systems often struggle with continuous, large-vocabulary recognition.
• Practical SLR requires robust methods for real-world applications.

Purpose of the Study:

• To develop a novel phonology- and radical-coded Chinese SLR framework.
• To demonstrate the feasibility of continuous SLR using accelerometer (ACC) and surface electromyography (sEMG) sensors.
• To achieve high recognition accuracy for a large vocabulary of Chinese characters.

Main Methods:

• Utilizing ACC and sEMG sensors for feature extraction.
• Employing a moving average algorithm for gesture segmentation.
• Implementing dynamic time warping (DTW) and hidden Markov models (HMM) for recognition.
• Classifying palm orientation, hand movement, and hand shape.

Main Results:

• Achieved 96.01% ± 0.83% accuracy for coded gesture recognition.
• Achieved 92.73% ± 1.47% accuracy for character recognition with 223 characters.
• Demonstrated sEMG signal consistency for hand shapes, independent of movement.
• Showcased scalability with minimal increase in training data for larger vocabularies.

Conclusions:

• The proposed SLR framework is effective for continuous Chinese character recognition.
• The system demonstrates high accuracy and scalability for practical applications.
• This research offers a promising new direction for developing practical Chinese SLR systems.