Toward Sensor Fusion Neuromuscular Interface for Continuous Finger Joint Angle Estimation via Deep Transfer Learning

Accurate decoding of motor intent from biosignals is an important step toward intuitive upper-limb prosthetic-control interfaces. We propose a novel high-dimensional multimodal deep learning framework that fuses surface electromyography (sEMG) and B-mode ultrasound (US) images to estimate metacarpophalangeal and proximal interphalangeal joint angles continuously. The framework employs a shared Encoder-Decoder-Regression architecture integrating convolutional neural networks (CNNs), transposed convolutions, an action-conditioned multi-head cross-attention module (ATT) that uses the commanded action as a query, and long short-term memory (LSTM) layers to jointly capture spatiotemporal features from both modalities. To improve cross-subject generalization and reduce data requirements for new users, we introduce a transfer learning strategy with parameter freezing. Experiments on data from seven able-bodied subjects show that, compared with sEMG-only and US-only baselines, the fusion model reduces test local root mean square error (RMSE) by 2.187° (23.385%) and 0.890° (11.054%), and increases test local correlation (Pearson's $r$ ) by 0.069 (10.02%) and 0.039 (5.48%) ( ${p}\lt{0}.{05}$ ), supporting the potential of multimodal fusion for future prosthetic-control interfaces. A preliminary validation on one amputee participant further supports the feasibility of applying the framework under a residual-limb sensing condition. Ablation studies further confirm that the full CNN+LSTM+ATT model achieves the best performance, reducing test local RMSE by 0.933° (11.524%) and increasing test local correlation by 0.033 (4.56%) ( ${p}\lt {0}.{05}$ ). Furthermore, fine-tuning the pretrained model with only 25% of a new subject's data yields performance comparable to full retraining, highlighting the framework's data efficiency.