Related Experiment Video
Updated: Jul 30, 2025
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
A Deep Q-Network based hand gesture recognition system for control of robotic platforms
Patricio J Cruz1,2, Juan Pablo Vásconez3, Ricardo Romero4
1Artificial Intelligence and Computer Vision Research Lab, Departamento de Informática y Ciencias de la Computación (DICC), Escuela Politécnica Nacional, Ladrón de Guevara, 170517, Quito, Ecuador. patricio.cruz@epn.edu.ec.
This study introduces a novel reinforcement learning (RL) approach for hand gesture recognition (HGR) using electromyography (EMG) and inertial measurement unit (IMU) signals. The developed system effectively controls robotic platforms, demonstrating high accuracy and fast response times.
Area of Science:
- Robotics and Human-Machine Interaction
- Machine Learning and Signal Processing
Background:
- Hand gesture recognition (HGR) systems utilizing electromyography (EMG) and inertial measurement unit (IMU) signals are crucial for advanced human-machine applications.
- Current state-of-the-art HGR systems predominantly employ supervised machine learning, leaving reinforcement learning (RL) approaches for human-machine interfaces as an open research area.
Purpose of the Study:
- To present a novel reinforcement learning (RL) approach for classifying EMG-IMU signals for hand gesture recognition (HGR).
- To develop and evaluate a Deep Q-learning (DQN) agent for real-time HGR and its application in controlling robotic systems.
Main Methods:
- Utilized a Myo Armband sensor to acquire synchronized EMG and IMU signals.
- Developed a Deep Q-learning (DQN) agent to learn a policy for classifying hand gestures from the acquired signals.
- Integrated the HGR system with PID controllers to command two robotic platforms: a helicopter test bench and a UR5 robot.
Main Results:
- Achieved high accuracy for classification ([Formula: see text]) and recognition ([Formula: see text]) with an average inference time of 20 ms per window.
- Demonstrated superior performance compared to existing methods in the literature.
- Successfully controlled both robotic platforms, showcasing the system's effectiveness, speed, and accuracy in real-world applications.
Conclusions:
- The proposed RL-based HGR system effectively classifies EMG-IMU signals with high accuracy and low latency.
- The developed DQN agent provides a robust method for real-time HGR, outperforming other approaches.
- The system's successful application in controlling robotic platforms highlights its potential for advanced human-machine interfaces.
