Related Experiment Video
Updated: Sep 4, 2025
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
Biosignal-based transferable attention Bi-ConvGRU deep network for hand-gesture recognition towards online upper-limb
Baao Xie1, James Meng2, Baihua Li3
1School of Electrical and Information Engineering, Tianjin University, China; Eastern Institute of Advanced Study, China.
Background And Objective:
Upper-limb amputation can significantly affect a person's capabilities with a dramatic impact on their quality of life. As a biological signal, surface electromyogram (sEMG) provides a non-invasive means to measure underlying muscle activation patterns, corresponding to specific hand gestures. This project aims to develop a real-time deep learning based recognition model to automatically and reliably recognise these complex signals of a wide range of daily hand gestures from amputees and non-amputees.
Methods:
This paper proposes an attention bidirectional Convolutional Gated Recurrent Unit (Bi-ConvGRU) deep neural network for hand-gesture recognition. By training on sEMG data from both amputees and non-amputees, the model can learn to recognise a group of fine-grained hand movements. This is a significantly more challenging and underexplored area, compared to existing studies on coarse-control in lower limbs. One dimensional CNNs are initially used to extract intra-channel features. The novel use of a bidirectional sequential GRU (Bi-GRU) deep neural network allows the exploration of correlation of muscle activation among multi-channel sEMG signals from both prior and posterior time sequences. Importantly, the attention mechanism is employed following Bi-GRU layers. This enables the model to learn vital parts and feature weights, increasing robustness to bio-data noise and irregularity. Finally, we introduce the first of its kind transfer learning, demonstrating that a baseline model pre-trained with non-amputee data can be effectively refined with amputee data to build a personalised model for amputees.
Results:
The attention Bi-ConvGRU was evaluated on the benchmark database Ninapro, and achieved an average accuracy of 88.7%, outperforming the state-of-the-art on 18 gesture recognition by 6.7%.
Conclusions:
To our knowledge, the developed end-to-end deep learning model is the first of its kind that enables reliable predictive decision making in short time windows (160ms). This reduced latency limits physiological awareness, enabling the potential for real-time, online and thus more intuitive bio-control of prosthetic devices for amputees.
