Related Experiment Video
Updated: Jun 23, 2025
06:37
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
Published on: December 15, 2023
2.7K
Video-Based Sign Language Recognition via ResNet and LSTM Network
Jiayu Huang1, Varin Chouvatut1
1Department of Computer Science, Faculty of Science, Chiang Mai University, Chiang Mai 50200, Thailand.
Journal of Imaging
|June 26, 2024
Summary
This study introduces a new sign language recognition method using Residual Network (ResNet) and Long Short-Term Memory (LSTM) for improved communication accessibility. The ResNet-LSTM model enhances spatio-temporal feature extraction, leading to higher accuracy in recognizing sign language actions.
Area of Science:
- Computer Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Sign language recognition technology aids communication for hearing-impaired individuals.
- Deep learning offers technical support for sign language recognition.
- Traditional methods using convolutional neural networks face challenges in feature extraction and computational resources.
Purpose of the Study:
- To present a novel video-based sign language recognition method.
- To improve the accuracy and efficiency of sign language recognition.
- To address the limitations of traditional methods in feature extraction and computational demands.
Main Methods:
- Utilized a Residual Network (ResNet) as the backbone for feature extraction.
- Employed Long Short-Term Memory (LSTM) networks to capture long sequence features.
- Combined ResNet for spatial feature extraction and LSTM for temporal feature learning from sign language videos.
Main Results:
- The proposed ResNet-LSTM method achieved an accuracy of 86.25% on the Argentine Sign Language (LSA64) dataset.
- Demonstrated superior performance with an F1-score of 84.98% and precision of 87.77%.
- Effectively extracted spatio-temporal features, significantly improving sign language action recognition rates.
Conclusions:
- The ResNet-LSTM method offers an effective solution for video-based sign language recognition.
- The approach enhances the extraction of spatio-temporal features, leading to improved recognition accuracy.
- This technology holds promise for better communication tools for the hearing-impaired community.
