Intelligibility of naturally produced and synthesized Mandarin speech by cochlear implant listeners

Mandarin is a tonal language, and it is important to preserve lexical tone information in synthesized speech. With natural speech, Chinese cochlear implant (CI) users have difficulty perceiving voice pitch cues important for lexical tone perception; it is unclear whether this difficulty persists in Mandarin synthesized speech. In this study, intelligibility of naturally produced and synthesized Mandarin speech was measured in Chinese CI listeners; intelligibility was also measured in a control group of normal-hearing (NH) listeners. Five synthesized voices were selected to represent different talker genders (male, female, child), speaking rates (normal, slow), and speaking styles (emotional, accent). The data showed that while modern Mandarin text-to-speech (TTS) systems can provide perfect speech intelligibility for NH listeners, overall intelligibility was much poorer for CI than for NH listeners. CI performance was significantly poorer with synthesized speech than with natural speech (p < 0.001). CI listeners were highly sensitive to the "extra-atypical" synthesized emotional and accented speech. Performance with each of the synthesized speech types was significantly correlated with performance with natural speech in CI users (p < 0.01 in all cases). While modern TTS systems offer educational and communication benefits to CI users and hearing-impaired individuals, the selection of synthesized voices should be carefully considered in education applications of TTS for hearing-impaired individuals, especially CI children, since poor intelligibility performance may affect language learning.