NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality.

Area of Science:

• Artificial Intelligence
• Speech Synthesis
• Machine Learning

Background:

• Recent advancements in text-to-speech (TTS) technology raise questions about achieving human-level quality.
• Defining and objectively measuring TTS quality remains a challenge.

Purpose of the Study:

• To define and establish criteria for human-level quality in text-to-speech systems.
• To develop a TTS system, NaturalSpeech, capable of achieving human-level speech quality.
• To validate the system's performance against human recordings using rigorous statistical methods.

Main Methods:

• Leveraging a variational auto-encoder (VAE) for end-to-end text-to-waveform generation.
• Implementing key modules: phoneme pre-training, differentiable duration modeling, bidirectional prior/posterior modeling, and a VAE memory mechanism.
• Establishing quality judgment guidelines based on statistically significant subjective measures.

Main Results:

• The NaturalSpeech system achieved a Comparative Mean Opinion Score (CMOS) of -0.01 on the LJSpeech dataset.
• Statistical analysis (Wilcoxon signed rank test) showed no significant difference between NaturalSpeech and human recordings (p >> 0.05).
• This represents the first time a TTS system has demonstrated statistically indistinguishable quality from human speech.

Conclusions:

• NaturalSpeech successfully achieves human-level text-to-speech quality.
• The study provides a robust framework for defining and evaluating TTS quality.
• The developed methods pave the way for more natural and human-like synthetic speech generation.