A Wireless, Battery-Free Artificial Throat Patch with Deep Learning for Emotional Speech Recognition

Technologies for sensing and translating throat signals into speech have advanced the communication for people with vocal impairments. However, the practical deployment of these technologies is currently hindered by tethered system, bulky components and the lack of emotional recognition capability. Here, we demonstrate a wireless, battery-free artificial throat patch system (ATPS) coupling with deep learning method to enable simultaneous recognition of speech and emotion. The sensing module of the ATPS integrates a carbon nanotube-based thin-film strain sensor and a miniaturized flexible printed circuit board (FPCB). Real-time sensing and transmission of throat signals is facilitated by a smartphone-linked system incorporating an FPCB with an embedded near-field communication antenna and low-power electronic components. Furthermore, the information-rich sensor readouts are further extracted and successfully demonstrated to be sufficient to emotional recognition of speech by implementing a hybrid deep learning architecture. This proposed system expands the arsenal of tools available for mute patients in speech recognition and communication and further realizes personalized clinical implementations.