Exploring Dysphonic Artificial Intelligence Voice Cloning for Speech Intelligibility in Noise

Area of Science:

• Speech Science
• Artificial Intelligence
• Medical Acoustics

Background:

• Voice cloning technology offers potential for studying rare voice disorders.
• This research investigates AI's ability to capture dysphonic speech intelligibility.

Purpose of the Study:

• To assess if AI-generated voice clones can replicate the intelligibility deficits associated with dysphonia.
• To evaluate the effectiveness of AI voice cloning in modeling dysphonic speech characteristics.

Main Methods:

• Generated AI voice clones for 12 speakers (6 with dysphonia, 6 healthy).
• Conducted three listener experiments evaluating natural vs. AI-generated speech perception and intelligibility.
• Utilized the Hearing-in-Noise Test for speech intelligibility measurements in noise.

Main Results:

• Listeners detected differences in dysphonic voices more readily than healthy ones.
• AI-generated dysphonic voices showed higher intelligibility than natural dysphonic voices.
• Male dysphonic speakers' intelligibility increased from 35.5% to 66.5% with AI cloning.

Conclusions:

• Current AI voice cloning tools do not accurately replicate the reduced intelligibility of dysphonic speech.
• Limitations exist in AI's capacity to model the acoustic features of voice disorders.
• AI voice cloning may not be suitable for studying the perceptual effects of dysphonia in its current state.