Surface Electromyography-Based Recognition, Synthesis, and Perception of Prosodic Subvocal Speech

Area of Science:

• Biomedical Engineering
• Speech and Hearing Science
• Rehabilitation Engineering

Background:

• Individuals with vocal impairments often face communication challenges.
• Existing assistive communication technologies may have limitations in naturalness and expressiveness.
• Surface electromyography (sEMG) offers a potential non-invasive method to capture articulatory muscle activity.

Purpose of the Study:

• To evaluate a novel communication system translating surface electromyographic (sEMG) signals into personalized synthetic speech.
• To assess the system's effectiveness in word recognition, prosodic (stress) classification, and listener perception.
• To compare synthesized speech with natural and electrolaryngeal (EL) speech.

Main Methods:

• Recorded sEMG signals from facial and neck muscles during sub vocalization (silent mouthing) of speech phrases.
• Utilized personalized voice synthesis to translate sEMG data into speech.
• Evaluated word recognition (96.3% accuracy) and prosodic stress classification (91.2% accuracy) using naïve listeners.

Main Results:

• The sEMG-based system demonstrated high accuracy in lexical content and prosodic stress recognition.
• Synthetic speech was rated as more acceptable and intelligible than electrolaryngeal (EL) speech.
• Natural speech achieved the highest ratings for acceptability, intelligibility, and prosodic stress discrimination.

Conclusions:

• This proof-of-concept study validates the feasibility of subvocal sEMG-based communication for lexical and prosodic functions.
• The system shows promise as an alternative communication method for individuals with vocal impairments and residual articulatory function.
• Further research can refine the technology for broader clinical application.