Improving Hand Gesture Recognition Robustness to Dynamic Posture Variations by Multimodal Deep Feature Fusion

Area of Science:

• Biomedical Engineering
• Robotics
• Human-Computer Interaction

Background:

• Surface electromyography (sEMG) shows potential for decoding motor intentions but faces challenges in real-world applications due to non-ideal factors.
• Existing gesture recognition methods struggle with variations in body posture, limiting their use in assistive robotics.

Purpose of the Study:

• To develop and validate a robust gesture recognition method for assistive robots that accounts for posture variations.
• To improve the accuracy and reliability of myoelectric control in dynamic, real-life scenarios.

Main Methods:

• Proposed a Multimodal Canonical Correlation Analysis Feature Fusion Classification (MCAFC) method.
• Extracted deep features from sEMG and acceleration signals using convolutional neural networks.
• Fused features using canonical correlation analysis and classified gestures with linear discriminant analysis.

Main Results:

• MCAFC achieved an average classification accuracy of 93.44% across multiple dynamic postures.
• Demonstrated an average motion completion rate of 94.05% and completion time of 1.38 seconds.
• Outperformed comparable methods in gesture recognition accuracy and robustness to posture variations.

Conclusions:

• The proposed MCAFC method is feasible and superior for gesture recognition with posture variations.
• This multimodal signal feature fusion offers a new, effective scheme for myoelectric control in assistive robotics.