An Extended Spatial Transformer Convolutional Neural Network for Gesture Recognition and Self-Calibration Based on...

Area of Science:

• Biomedical Engineering
• Human-Computer Interaction
• Machine Learning

Background:

• Surface electromyography (sEMG) is crucial for human-machine interaction but suffers accuracy loss due to electrode shift.
• Accurate self-calibration of virtual hand posture is essential for immersive applications like virtual reality (VR).

Purpose of the Study:

• To develop an armband system integrating sEMG and IMU for robust gesture recognition and self-calibration.
• To introduce an Extended Spatial Transformer Convolutional Neural Network (EST-CNN) for one-shot processing of gesture recognition and posture estimation.

Main Methods:

• An armband fusing sEMG and IMU sensors with autonomously adjustable gain was developed.
• An EST-CNN incorporating feature-enhanced pretreatment (FEP), spatial transformer layers (STL), and a fine-tuning layer (FTL) was designed.
• The system performs gesture recognition and self-calibration by automatically learning transformation relations and estimating rotational angles.

Main Results:

• The EST-CNN achieved 97.06% accuracy in gesture recognition under electrode shift conditions, outperforming standard CNN by 5.81%.
• The system demonstrated a 99.44% fitness between estimated and true rotational angles, enabling high-resolution posture estimation.
• The proposed method allows for non-discretized angle calculation and effective self-calibration.

Conclusions:

• The novel armband and EST-CNN system significantly enhance the robustness and accuracy of sEMG-based gesture recognition and self-calibration.
• This approach offers a promising solution for reliable human-machine interaction, particularly in dynamic environments with potential sensor displacement.
• The ability to perform one-shot processing for both recognition and calibration simplifies user experience and improves system adaptability.