Related Experiment Video
Updated: Mar 27, 2026
11:15
fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
Published on: May 23, 2017
7.7K
Soft Multiaxial Strain Mapping Interface with AI-Driven Decoding for Silent Speech in Noise.
Sunguk Hong1, Junyoung Yoo2, Sung-Min Park1,2,3,4,5,6
1Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea.
Cyborg and Bionic Systems (Washington, D.C.)
|March 25, 2026
Summary
This study introduces a novel silent speech interface (SSI) using computer vision to detect throat muscle strain, reconstructing clear speech even in noisy environments. This innovation offers a reliable alternative for communication where traditional microphones fail.
Area of Science:
- Biomedical Engineering
- Signal Processing
- Machine Learning
Background:
- Traditional microphones struggle in noisy environments.
- Silent speech interfaces (SSIs) offer an alternative for clear audio capture.
- Existing wearable sensors for SSIs have limitations in scalability and reliability.
Purpose of the Study:
- To develop a reconceptualized SSI using computer vision to monitor throat muscle strain.
- To enable clear alphabetic communication via voice reconstruction under extreme noise conditions.
- To enhance the robustness and applicability of SSIs in real-world scenarios.
Main Methods:
- Integration of a computer vision-based optical strain (CVOS) sensor with deep learning for voice reconstruction.
- Utilizing a soft silicone substrate with micromarkers and a camera for high-sensitivity strain pattern capture.
- Implementing physics-based automated baseline calibration and content-adaptive temporal attention for robust inference.
Main Results:
- The CVOS sensor demonstrated high-sensitivity marker detection and reliable strain pattern capture.
- The CVOS-based SSI achieved clear alphabetic communication in extreme noise conditions.
- Real-time adaptive signal processing compensated for anatomical variability, ensuring robustness.
Conclusions:
- The proposed CVOS-based SSI offers a scalable, reliable, and practical solution for clear speech communication in noisy environments.
- The synergy of optimized algorithms and interface design enables effective alphabet-based communication.
- The system's performance was validated in real-world noisy scenarios, confirming its practical applicability.
