A Wireless, Battery-Free Artificial Throat Patch with Deep Learning for Emotional Speech Recognition

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Wearable Technology

Background:

• Current speech-enabling technologies for vocal impairments are often limited by tethers, bulkiness, and a lack of emotional recognition.
• Addressing these limitations is crucial for improving communication accessibility and quality of life for affected individuals.

Purpose of the Study:

• To develop a wireless, battery-free artificial throat patch system (ATPS) capable of simultaneous speech and emotion recognition.
• To overcome the practical deployment challenges of existing assistive communication technologies.

Main Methods:

• Integration of a carbon nanotube-based thin-film strain sensor and a miniaturized flexible printed circuit board (FPCB) into the ATPS.
• Utilizing a smartphone-linked system with near-field communication (NFC) for real-time signal transmission.
• Implementing a hybrid deep learning architecture for extracting speech and emotion information from sensor data.

Main Results:

• Demonstrated successful real-time sensing and wireless transmission of throat signals.
• Achieved accurate simultaneous recognition of speech and emotion using the ATPS and deep learning.
• Validated the sufficiency of sensor readouts for emotional recognition.

Conclusions:

• The developed ATPS offers a wireless, battery-free solution for speech and emotion recognition, significantly advancing assistive communication.
• This technology expands communication tools for individuals with vocal impairments and enables personalized clinical applications.
• The system represents a significant step towards more natural and expressive communication for those unable to speak conventionally.