Toward Sensor Fusion Neuromuscular Interface for Continuous Finger Joint Angle Estimation via Deep Transfer Learning.

Area of Science:

• Biomedical Engineering
• Machine Learning
• Neuroscience

Background:

• Decoding motor intent is crucial for advanced prosthetic control.
• Current interfaces often rely on single biosignal modalities, limiting performance.
• High-dimensional multimodal fusion offers a promising avenue for improved decoding accuracy.

Purpose of the Study:

• To develop and evaluate a novel deep learning framework for continuous joint angle estimation.
• To fuse surface electromyography (sEMG) and B-mode ultrasound (US) data for enhanced motor intent decoding.
• To investigate the efficacy of transfer learning for improving cross-subject generalization and reducing data requirements.

Main Methods:

• A shared Encoder-Decoder-Regression deep learning architecture was employed.
• The framework integrated convolutional neural networks (CNNs), transposed convolutions, multi-head cross-attention (ATT), and long short-term memory (LSTM) layers.
• Transfer learning with parameter freezing was utilized to enhance generalization.

Main Results:

• The multimodal fusion model significantly outperformed sEMG-only and US-only baselines in estimating joint angles.
• Fusion reduced root mean square error (RMSE) by up to 23.385% and increased correlation by up to 10.02%.
• The full CNN+LSTM+ATT model demonstrated superior performance, and transfer learning achieved comparable results with 25% of the data.

Conclusions:

• Multimodal fusion of sEMG and US shows significant potential for intuitive upper-limb prosthetic control.
• The proposed framework demonstrates high accuracy and data efficiency, facilitating faster adaptation for new users.
• Preliminary validation suggests feasibility for amputee applications and residual-limb sensing.