Multimodal optimal matching and augmentation method for small sample gesture recognition.

Area of Science:

• Human-Computer Interaction
• Biomedical Engineering
• Machine Learning

Background:

• Gesture recognition using physiological signals offers natural interaction but requires extensive user-specific data.
• Current methods like transfer learning face challenges with negative transfer and limited data diversity.
• Collecting data from non-healthy users for sEMG-based models is particularly burdensome.

Purpose of the Study:

• To develop an efficient multimodal method for small-sample gesture recognition using surface electromyography (sEMG) and motion information.
• To reduce the burden of data acquisition for users, especially those with physical limitations.
• To enhance the accuracy and diversity of training samples for deep learning models in gesture recognition.

Main Methods:

• Proposed a multimodal optimal matching and augmentation method integrating motion information with sEMG signals.
• Utilized an optimal matching signal selection module to minimize inter-domain differences for new users.
• Implemented a similarity calculation augmentation module to increase training set diversity and modal-type embedding for enhanced information interaction.

Main Results:

• Achieved high accuracies of 93.69% on a self-collected stroke patient dataset, 91.65% on Ninapro DB1, and 98.56% on Ninapro DB5.
• Demonstrated effective gesture recognition with only one data acquisition per gesture.
• Showcased performance comparable to traditional models while significantly reducing data collection requirements.

Conclusions:

• The proposed multimodal approach significantly enhances small-sample gesture recognition using sEMG and motion data.
• This method offers a practical solution for efficient and accurate gesture recognition, particularly beneficial for non-healthy users.
• The findings pave the way for more accessible and less data-intensive human-computer interaction systems.