Exploring Dysphonic Artificial Intelligence Voice Cloning for Speech Intelligibility in Noise

Voice cloning technologies powered by artificial intelligence (AI) may provide new opportunities to study the perceptual effects of dysphonia, particularly in rare voice disorders where clinical data are scarce. This study examined whether AI-generated voice clones can capture the intelligibility characteristics of dysphonic speech.

Researchers generated voice clones for 12 speakers, including six speakers with dysphonia and six vocally healthy speakers. Sixty-four listeners evaluated both natural and AI-produced speech samples across three experiments. In Experiment 1, listeners judged whether samples were real or AI-generated. Experiment 2 focused on listeners' ability to identify AI-generated voices in paired comparisons. Experiment 3 measured speech intelligibility in noise using the Hearing-in-Noise Test.

In Experiment 1, listeners were more sensitive to differences in dysphonic voices compared with healthy voices. In Experiment 2, participants performed better when comparing pairs of real voices, achieving 66.8% accuracy. In Experiment 3, AI-generated dysphonic voices were more intelligible than natural dysphonic voices. Male dysphonic speakers demonstrated the largest increase in speech intelligibility scores, rising from 35.5% for real speech to 66.5% for AI-generated speech. Female speakers also showed consistent or enhanced intelligibility, with AI-generated dysphonic female voices reaching 67.0%.

Overall, findings indicate that current AI voice cloning tools fail to replicate the reduced intelligibility of dysphonic speech, suggesting limitations in accurately modeling the acoustic features of voice disorders.