Cross-Speaker Training and Adaptation for Electromyography-to-Speech Conversion

Area of Science:

• Speech synthesis
• Biomedical engineering
• Machine learning

Background:

• Surface Electromyography (EMG) signals from articulatory muscles can synthesize speech using Electromyography-to-Speech (ETS) models.
• Current ETS models improve synthesis by using multiple recordings from single speakers.

Purpose of the Study:

• To evaluate if using recordings from multiple speakers enhances ETS model performance.
• To assess the adaptability of cross-speaker ETS models to new speakers with limited data.

Main Methods:

• The EMG-Vox corpus, containing EMG and audio signals from four speakers across five sessions, was recorded.
• Cross-speaker and single-speaker ETS models were compared.
• Speaker adaptation experiments were conducted using cross-speaker models.

Main Results:

• Cross-speaker models demonstrated significantly better performance on average than single-speaker models.
• The performance improvement was attributed to the larger training dataset.
• Speaker adaptation from cross-speaker models resulted in higher synthesis quality compared to training from scratch.

Conclusions:

• Cross-speaker ETS models outperform single-speaker models in speech synthesis.
• Speaker adaptation of cross-speaker models is effective for unseen speakers, achieving comparable or better results than session adaptation.