Chinese sign language recognition based on surface electromyography and motion information

Area of Science:

• Computer Science
• Biomedical Engineering
• Linguistics

Background:

• Sign language recognition (SLR) faces challenges due to complex gestures and hand movement trajectories.
• Existing methods often struggle to capture the intricate structural features of sign language.

Purpose of the Study:

• To develop and evaluate a novel method for Chinese sign language recognition.
• To improve the accuracy and practicability of SLR by integrating multiple data sources.

Main Methods:

• Utilized surface electromyography (sEMG) signals and acceleration data for vocabulary segmentation.
• Employed a multi-sensor decision fusion method with a coupled hidden Markov model for recognition.
• Selected 120 common Chinese sign language words involving 9 gestures and 8 movement trajectories.

Main Results:

• Achieved an average recognition rate of 90.41% for Chinese sign language vocabulary.
• Demonstrated the effectiveness of fusing sEMG signal amplitude states and motion trajectory information.
• Validated the practicability of the proposed fusion method in sign language recognition.

Conclusions:

• The fusion of sEMG signals and motion information offers a robust and practical approach to sign language recognition.
• The coupled hidden Markov model-based multi-sensor fusion effectively addresses the complexities of sign language.
• This research contributes to advancing the field of automated sign language understanding.