Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals

Area of Science:

• Human-Computer Interaction
• Wearable Technology
• Biomedical Signal Processing

Background:

• Gesture recognition is crucial for human-computer interaction (HCI).
• Wearable sensors offer robust and convenient gesture recognition compared to computer vision methods.
• Existing datasets lack multi-category gesture data, limiting deep learning model performance.

Purpose of the Study:

• To create a comprehensive multi-category gesture dataset using wearable surface electromyography (sEMG) and inertial measurement unit (IMU) signals.
• To develop and evaluate deep learning models for accurate multi-gesture classification.
• To improve recognition accuracy for complex HCI scenarios.

Main Methods:

• Recorded a new dataset of 20 distinct gestures using a wearable device capturing sEMG and IMU signals.
• Experimented with baseline deep learning models: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Transformer.
• Developed improved models including CNN-Res, LSTM-Res, GRU-Res, and Transformer-CNN by integrating residual networks and model fusion.

Main Results:

• Baseline CNN and RNN models achieved over 95% accuracy; Transformer model achieved 71.68%.
• The CNN-Res model reached 98.24% accuracy with reduced training/testing times.
• LSTM-Res and GRU-Res models achieved the highest accuracies at 99.67% and 99.49%, respectively.
• The fused Transformer-CNN model significantly improved Transformer performance to 98.96%.

Conclusions:

• The new sEMG and IMU dataset supports advanced multi-gesture recognition research.
• Deep learning models, particularly hybrid approaches like LSTM-Res, GRU-Res, and Transformer-CNN, demonstrate high accuracy for multi-category gesture classification.
• These findings pave the way for more sophisticated and reliable gesture-based HCI systems.