Audio-visual feedback for electromyographic control of vowel synthesis

Area of Science:

• Neuroscience
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Surface electromyography (sEMG) offers potential for non-invasive human-machine interfaces.
• Current sEMG interfaces often rely on overt muscle activity and visual feedback, limiting their mobility and discretion.
• Developing covert sEMG control is crucial for discreet and mobile communication aids.

Purpose of the Study:

• To design and test a human-machine interface utilizing covertly recorded sEMG for speech synthesis control.
• To evaluate the learnability and generalizability of controlling vowel synthesis using auditory feedback alone.

Main Methods:

• sEMG signals were recorded from the Auricularis Posterior muscle in 5 healthy participants.
• Participants underwent multiple training sessions to transition from audio-visual feedback to auditory feedback only for vowel synthesis.
• The interface used sEMG to control synthesized vowel sounds.

Main Results:

• Participants successfully learned to control vowel synthesis using only auditory feedback and sEMG signals.
• An average accuracy of 67% was achieved in controlling vowel synthesis.
• The learned skill generalized to new, untrained vowel targets.

Conclusions:

• Covertly recorded sEMG can be effectively used to control speech synthesis.
• Auditory feedback alone is sufficient for learning and performing sEMG-based vowel synthesis.
• This technology represents a significant step towards developing more reliable and discreet mobile communication interfaces.