Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Area of Science:

• Biomedical Engineering
• Machine Learning
• Human-Computer Interaction

Background:

• Surface electromyography (sEMG) signals are crucial for human-computer interaction (HCI).
• Traditional machine learning methods face challenges in feature selection for accurate gesture recognition.
• Neural networks offer strong nonlinear generalization capabilities for complex pattern recognition.

Purpose of the Study:

• To develop an advanced deep learning model for robust multi-gesture recognition.
• To improve the accuracy and efficiency of gesture recognition systems by fusing multi-source data.
• To enhance the performance of HCI systems, particularly for exoskeleton robots and prosthetic control.

Main Methods:

• Proposed a two-stream residual network incorporating an attention mechanism.
• Processed both surface EMG and hand acceleration signals in separate network branches.
• Utilized segmented networks for comprehensive feature extraction and an attention mechanism to focus on relevant information.
• Fused deep features from both streams for improved multi-gesture recognition accuracy.

Main Results:

• Achieved a recognition accuracy of 88.25% for 49 gestures on the NinaPro DB2 dataset.
• Demonstrated effective capture of gesture-specific features, leading to enhanced accuracy and robustness.
• The attention mechanism successfully prioritized crucial information, improving model performance.

Conclusions:

• The proposed two-stream residual network with attention is effective for accurate and robust gesture recognition.
• Multi-source information fusion significantly enhances the performance of gesture recognition models.
• This approach holds promise for real-time control of exoskeleton robots and myoelectric prosthetics, improving user experience.