Related Experiment Video
Updated: Nov 29, 2025
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
sEMG-Based Neural Network Prediction Model Selection of Gesture Fatigue and Dataset Optimization
Fujun Ma1, Fanghao Song2, Yan Liu2
1Center for Advanced Jet Engineering Technologies (CaJET), Key Laboratory of High-efficiency and Clean Mechanical Manufacture (Ministry of Education), National Experimental Teaching Demonstration Center for Mechanical Engineering (Shandong University), School of Mechanical Engineering, Shandong University, Jinan 250061, China.
Predicting gesture fatigue is crucial for human-machine interaction. This study uses surface electromyography (sEMG) signals and artificial neural networks (ANNs), finding Long Short-Term Memory (LSTM) models effective for predicting integrated gesture fatigue.
Area of Science:
- Human-Computer Interaction
- Biomedical Engineering
- Signal Processing
Background:
- Fatigue energy consumption in gestures is typically analyzed using surface electromyography (sEMG) power spectrum.
- Existing research primarily addresses independent gestures, neglecting the integrated nature of real-world operations.
- Understanding integrated gesture fatigue is vital for optimizing user experience in human-machine interaction.
Purpose of the Study:
- To predict the fatigue degree of integrated gestures by training neural networks on independent gesture data.
- To evaluate the efficacy of different artificial neural networks (ANNs) for gesture fatigue prediction.
- To determine optimal datasets for predicting one-handed versus two-handed gesture fatigue.
Main Methods:
- Decomposed nine natural gestures (browsing, gaming, typing) into independent gestures to calculate energy consumption.
- Trained and compared Backpropagation (BP), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM) models.
- Utilized Support Vector Machine (SVM) for verification and Mean Square Error (MSE), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE) for evaluation.
Main Results:
- The Long Short-Term Memory (LSTM) model demonstrated superior performance in predicting gesture fatigue.
- Processed sEMG signals are suitable for training datasets predicting one-handed gesture fatigue.
- Wavelet decomposition coefficients are more effective for predicting high-dimensional sEMG signals of two-handed gestures.
Conclusions:
- LSTM models offer a promising approach for predicting integrated gesture fatigue.
- The choice of dataset (processed sEMG vs. wavelet coefficients) impacts prediction accuracy based on gesture complexity (one-handed vs. two-handed).
- Findings can enhance human-machine interactive gesture design, prevent overuse, and improve user experience.
