Perception of Synthesized Mandarin Speech Based on a Large-Scale Language Model Among Deaf Adults With Cochlear...

Area of Science:

• Speech processing
• Auditory perception
• Artificial intelligence

Background:

• Cochlear implants (CIs) offer hearing restoration but speech intelligibility remains a challenge.
• Natural speech processing is complex, and synthetic speech offers potential for tailored auditory rehabilitation.

Purpose of the Study:

• To compare the intelligibility of large language model (LLM)-based synthetic Mandarin speech versus natural speech for adults with CIs.
• To investigate processing characteristics, including reaction times and variability, in CI users and normal-hearing (NH) listeners.

Main Methods:

• Fifty Mandarin speakers (25 with CIs, 25 NH) participated.
• Sentence recognition rates and reaction times were measured using natural and LLM-based synthetic speech (male/female voices, slow/normal rates).
• Linear mixed-effect models analyzed listener group and speech type effects.

Main Results:

• CI users showed lower recognition rates and longer reaction times than NH listeners for both speech types.
• Individual variability was greater in CI users.
• LLM-based synthetic speech was equally intelligible as natural speech for CI users when speaking rates were matched.

Conclusions:

• LLM-based synthetic speech can achieve natural speech-level intelligibility for CI recipients at comparable speaking rates.
• This technology has significant implications for speech perception assessment and auditory rehabilitation for CI users.
• Potential for developing personalized training materials using advanced text-to-speech models.