Related Experiment Video
Updated: Aug 9, 2026
06:32
Bringing the Clinic Home: An At-Home Multi-Modal Data Collection Ecosystem to Support Adaptive Deep Brain Stimulation
Published on: July 14, 2023
A Portable Sign Language Collection and Translation Platform with Smart Watches Using a BLSTM-Based Multi-Feature
Zhenxing Zhou1, Vincent W L Tam1, Edmund Y Lam1
1Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam, Hong Kong, China.
Micromachines
|February 25, 2022
Summary
This study introduces a novel framework using two smartwatches and a bidirectional long short-term memory (BLSTM) model for precise continuous sign language recognition (CSLR). The system significantly reduces word error rates, enabling real-time communication for sign language users.
Area of Science:
- Sensor technology and artificial intelligence for human-computer interaction.
- Development of efficient algorithms for real-time continuous sign language recognition (CSLR).
Background:
- Vision-based CSLR methods are computationally intensive and prone to delays.
- Gesture-based CSLR offers efficiency but suffers from limited sensor data.
- Bridging the information gap in wearable sensor data is crucial for accurate CSLR.
Purpose of the Study:
- To propose a multi-feature framework for precise gesture-based CSLR using wearable sensors.
- To enhance the performance of continuous sign language recognition by leveraging diverse input features.
- To develop a portable platform for real-time sign language data collection and translation.
Main Methods:
- Utilizing a bidirectional long short-term memory (BLSTM) network.
- Extracting multiple sets of input features from gesture data captured by two smartwatches.
- Testing the framework on a new Hong Kong sign language (HKSL) dataset.
Main Results:
- The proposed BLSTM-based multi-feature framework achieved a significantly lower word error rate compared to existing methods.
- Demonstrated the effectiveness of multi-feature extraction for improving gesture-based CSLR accuracy.
- Successfully developed a portable platform for real-time sign language recognition.
Conclusions:
- The multi-feature BLSTM framework effectively addresses the limitations of insufficient sensor data in gesture-based CSLR.
- The developed platform offers a practical solution for breaking communication barriers for sign language users.
- This research advances the field of sensor technology for real-time, accessible communication tools.
