Perceptual Adaptation to Noise-Vocoded Voice Clones

Voice cloning technology has developed rapidly, and current synthesis techniques can produce humanlike voices. Recent research has shown that cloned voices can now be more intelligible than their human originals in background noise, a signal-extrinsic degradation. Here, we aimed to establish whether this intelligibility benefit for voice clones degraded using noise-vocoding, a signal-intrinsic manipulation that severely degrades fine-grained spectrotemporal and harmonic information in the speech signal. We also investigated whether listeners can perceptually adapt to noise-vocoded voice clones. We compared the intelligibility of ten six-band noise-vocoded voice clones with their human originals. Listeners heard 80 sentences, in two counterbalanced blocks: a block of 40 sentences by voice clones and a block of 40 sentences by human, to compare perceptual adaptation to both speech types. Listeners showed 13.2% higher accuracy for voice clones. Overall, listeners adapted to the noise-vocoded speech and improved by around 16% over the course of 40 sentences in the first block only, regardless of whether voices were human or cloned. The intelligibility benefit for cloned voices thus persisted for noise-vocoded speech, and listeners adapted to noise-vocoded cloned voices as much as for noise-vocoded human voices. Our results have implications for applications of voice clones, in particular in future development of assistive listening technologies.