Related Experiment Video
Updated: Sep 13, 2025
08:25
Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.1K
Low-cost computation for isolated sign language video recognition with multiple reservoir computing.
A R Syulistyo1,2, Y Tanaka1,3, D Pramanta4
1Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology 2-4 Hibikino, Wakamatsu, Kitakyushu, Japan.
Plos One
|July 30, 2025
Summary
This study introduces a cost-effective sign language recognition system using reservoir computing (RC) for edge devices. The novel approach achieves high accuracy and significantly reduces training time, enhancing accessibility for the hearing-impaired community.
Area of Science:
- Computer Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Sign language recognition (SLR) systems are crucial for communication accessibility but face challenges with portability and computational demands on edge devices.
- Deep neural networks, while powerful, are often too computationally intensive for resource-constrained, server-independent applications.
- Reservoir computing (RC) offers a computationally efficient alternative suitable for edge device implementation.
Purpose of the Study:
- To develop a cost-effective sign language recognition (SLR) system optimized for edge devices with limited resources.
- To leverage reservoir computing (RC) for efficient and portable SLR implementation.
- To enhance RC performance through multiple reservoirs and MediaPipe preprocessing.
Main Methods:
- Implemented a novel sign language recognition (SLR) system utilizing reservoir computing (RC) with multiple reservoirs featuring distinct leak rates.
- Employed MediaPipe for preprocessing sign language videos, extracting and normalizing body and hand keypoint coordinates.
- Integrated keypoint extraction and normalization to improve robustness against background variations and signer positioning.
Main Results:
- The proposed Multiple Reservoir Computing (MRC) system achieved competitive accuracies (60.35% top-1, 84.65% top-5, 91.51% top-10) on the WLASL100 dataset.
- Outperformed deep learning models like Pose-TGCN and Pose-GRU in recognition accuracy.
- Demonstrated significantly reduced training time (52.7s) compared to traditional deep learning approaches (e.g., I3D at 20h).
Conclusions:
- The integration of MediaPipe preprocessing and multiple reservoirs in RC provides a robust and efficient solution for edge-based SLR.
- The proposed system offers a viable, cost-effective alternative to deep neural networks for portable sign language recognition.
- This approach enhances accessibility and privacy for hearing-impaired communities by enabling server-independent SLR operation.
