Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

Area of Science:

• Biomedical Engineering
• Human-Computer Interaction
• Wearable Technology

Background:

• Silent speech interfaces aim to restore communication for individuals with voice disorders and enable communication in challenging acoustic environments.
• Existing silent speech technologies often suffer from bulkiness, obtrusiveness, low accuracy, and susceptibility to interference.
• There is a need for unobtrusive, accurate, and robust silent speech recognition systems.

Purpose of the Study:

• To develop a wireless, unobtrusive, and robust silent speech interface.
• To track and decode speech-relevant movements of the temporomandibular joint for silent speech recognition.
• To demonstrate the potential of the developed interface in assistive technology and human-machine interactions.

Main Methods:

• A single soft magnetic skin sensor is placed behind the ear for data acquisition.
• Machine learning-based signal processing techniques are employed for speech decoding.
• The system is evaluated for speech recognition accuracy and robustness against environmental noise and user motion.

Main Results:

• The developed silent speech interface achieves high accuracy: 93.2% for phonemes and 87.3% for words.
• The system demonstrates robustness against ambient noise and daily user motions.
• The interface is wireless, unobtrusive, and socially acceptable, overcoming limitations of existing face-worn sensors.

Conclusions:

• The proposed soft magnetic skin-based silent speech interface offers a promising solution for restoring communication and enhancing human-machine interaction.
• The system's unobtrusiveness, accuracy, and robustness make it suitable for assistive technology applications.
• Future applications include silent speech-enabled smartphone assistants and drone control systems.