IMU-weighted multimodal fusion for robust gesture recognition across arm positions

Objective. Human-machine interfaces (HMIs) based on surface electromyography (sEMG) and A-mode ultrasound have achieved high gesture recognition accuracy; however, most studies have been conducted under a single static arm position. In practice, arm position variation degrades performance and limits translation beyond fixed-position laboratory settings. Although A-mode ultrasound can outperform sEMG at fixed positions, it is more sensitive to positional changes, an issue that has rarely been examined. This study addresses the cross-position sensitivity of ultrasound-based HMIs by fusing ultrasound data with sEMG and inertial measurement unit (IMU) data.Approach. We propose an IMU-weighted multimodal fusion framework. A feature fusion model and an sEMG single-modality model are trained in parallel. At the prediction stage, the arm position is estimated from the IMU, and the angular offset relative to the training positions is used to compute a continuous weight for combining the two models' posterior probabilities. Near training positions, greater weight is placed on the high-accuracy fusion model; as the offset grows, the contribution progressively shifts toward the robust sEMG model to avoid further degradation caused by A-mode ultrasound.Main results. Experiments were conducted across nine arm positions ranging from forearm elevation to lowering. With only two training positions, the proposed IMU-weighted method achieved an average accuracy of 85.88% across all nine positions, significantly outperforming other models.Significance. The method effectively exploits the high static accuracy of A-mode ultrasound and the cross-position stability of sEMG. With a low training burden, the proposed method improves cross-arm-position decoding accuracy, providing a practical route toward robust gesture recognition based on A-mode ultrasound across varying arm positions.