Spatiotemporal Dual-Channel Interpretable Hybrid Neural Network for HD-sEMG-Based Gesture Recognition

Accurate gesture recognition is crucial for precision control of upper limb prostheses. High-density surface electromyography (HD-sEMG) enhances spatial resolution and information richness of human gesture representation, thus improving myoelectric control of bionic limbs. Recently, deep learning has been increasingly applied to HD-sEMG to enhance gesture recognition performance. However, the black-box nature of neural networks limits their interpretability and model optimization, hindering their practical application. In this paper, we developed a spatiotemporal dual-channel interpretable hybrid neural network (STDC-Net), and validated it using the Capgmyo DB-a dataset. STDC-Net uses Feature Channels and Spatial Channels to process the feature and spatial information of sEMG signals respectively for increased interpretability. SHapley Additive exPlanations (SHAP) values are used to rank the feature importance, aiding feature filtering and reducing the impact of irrelevant features. Graph attention layers are used to calculate the connections between each Spatial Channel, illustrating the relationships between channels. Our results demonstrated the superior performance of our STDC-Net compared to state-of-the-art (SOTA) methods. STDC-Net achieved 99.8% accuracy with a 150 ms sliding window, exceeding real-time implementation requirements for intra-subject tasks. It reached an accuracy of 97.33 ± 2.53% after fine-tuning for inter-subject tasks, outperforming the SOTA methods. Importantly, the SHAP value maps and the channel connection maps enhance the interpretability of the neural networks by offering detailed insights into the contribution of input features and parameter interactions of the network. These findings suggest STDC-Net holds significant promise for real-time prosthetic control.