emg2vec: Self-supervised Pretraining in Electromyography-based Silent Speech Interfaces.

Area of Science:

• Biomedical Engineering
• Computer Science
• Speech Technology

Background:

• Silent speech interfaces (SSI) offer communication solutions for individuals with vocal impairments.
• Electromyography (EMG) is a promising signal source for SSI due to its non-invasiveness and effectiveness.
• Existing EMG-based SSI methods often require substantial labeled data for training.

Purpose of the Study:

• To propose and evaluate emg2vec, a novel self-pretraining framework for EMG-based SSI.
• To enhance both EMG-to-speech and EMG-to-text conversion capabilities.
• To demonstrate the benefits of self-pretraining over traditional supervised learning.

Main Methods:

• Development of the emg2vec self-pretraining framework.
• Implementation of EMG-to-speech and EMG-to-text conversion models.
• Experimental comparison of self-pretraining against training from scratch using varying amounts of labeled data.

Main Results:

• Self-pretraining with emg2vec significantly improved speech recognition word error rate (WER) by 7.32% (full dataset) and 5.18% (20% data).
• Speech synthesis also showed improvement, with a 2.91% gain when using 20% of the training data.
• The framework demonstrated superior performance compared to plain supervised learning.

Conclusions:

• The emg2vec self-pretraining framework enhances the performance of EMG-based silent speech interfaces.
• Self-pretraining is particularly beneficial for improving model accuracy when labeled data is scarce.
• This approach offers a more efficient and effective method for developing EMG-to-speech and EMG-to-text systems.