TENG-Based Self-Powered Silent Speech Recognition Interface: from Assistive Communication to Immersive AR/VR...

Area of Science:

• Biomedical Engineering
• Human-Computer Interaction
• Materials Science

Background:

• Silent speech communication is a vital area for individuals with speech impairments.
• Accurate real-time acquisition and decoding of jaw movements for silent speech recognition remain challenging.
• Existing methods lack the sensitivity and precision required for reliable silent speech decoding.

Purpose of the Study:

• To develop a real-time silent speech recognition system.
• To integrate a triboelectric nanogenerator-based flexible pressure sensor (FPS) with a deep learning framework.
• To enable contactless control of devices through decoded silent speech.

Main Methods:

• Utilized a porous pyramid-structured silicone film in an FPS for sensitive low-force pressure detection.
• Developed a convolutional neural network-long short-term memory (CNN-LSTM) hybrid network for signal decoding.
• Integrated the FPS and CNN-LSTM model for real-time silent speech acquisition and recognition.

Main Results:

• The FPS demonstrated high sensitivity in detecting low-force jaw movements (1 V N⁻¹ for 0-10 N and 4.6 V N⁻¹ for 10-24 N).
• The CNN-LSTM model achieved 95.83% classification accuracy for 30 daily word categories.
• Decoded silent speech signals were successfully translated into executable smartphone commands.

Conclusions:

• The proposed system offers a promising solution for silent speech recognition and communication.
• The integration of advanced sensors and deep learning enables precise contactless human-machine interaction.
• Potential applications include AR/VR environments and assistive technologies for individuals with speech impairments.