Related Experiment Video
Updated: Jan 2, 2026
08:15
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
1.1K
Dynamic Hand Gesture Recognition Using 3DCNN and LSTM with FSM Context-Aware Model.
Noorkholis Luthfil Hakim1, Timothy K Shih1, Sandeli Priyanwada Kasthuri Arachchi1
1Department of Computer Science and Information Engineering, National Central University, Taoyuan 32001, Taiwan.
Sensors (Basel, Switzerland)
|December 15, 2019
Summary
This study introduces a novel gesture control system for Smart TVs, integrating multiple applications for intuitive interaction. The deep learning model achieved high accuracy in recognizing gestures, enhancing user experience.
Area of Science:
- Human-Computer Interaction
- Computer Vision
- Artificial Intelligence
Background:
- The proliferation of Smart TV technology necessitates intuitive control methods beyond traditional remotes.
- Integrating diverse applications like media, social, and communication tools enhances Smart TV utility.
- Natural gesture control offers a user-friendly interface for complex Smart TV environments.
Purpose of the Study:
- To develop a unified gesture-based control system for a comprehensive Smart TV application suite.
- To design and implement a robust gesture recognition model for natural user interaction.
- To evaluate the system's performance in real-time application control.
Main Methods:
- A dataset of 24 static and dynamic gestures using RGB and depth images was collected.
- A deep learning architecture combining 3D Convolutional Neural Network (3DCNN) and Long Short-Term Memory (LSTM) was employed.
- A Finite State Machine (FSM) was integrated to manage application context and refine gesture classification.
Main Results:
- The combined RGB and depth data achieved 97.8% accuracy for eight selected gestures.
- The Finite State Machine (FSM) improved real-time gesture recognition performance from 89% to 91%.
- The system demonstrated effective integration of multiple Smart TV applications via gesture control.
Conclusions:
- The proposed gesture recognition system offers a natural and efficient way to interact with integrated Smart TV applications.
- The combination of 3DCNN, LSTM, and FSM provides a powerful framework for spatio-temporal gesture analysis.
- This approach significantly enhances the user experience on Smart TV platforms.
