CMDF-TTS: Text-to-speech method with limited target speaker corpus.

Area of Science:

• Speech synthesis
• Machine learning
• Digital signal processing

Background:

• End-to-end Text-to-Speech (TTS) models require large auxiliary corpora, increasing training costs.
• Existing methods struggle with high-quality synthesis using limited target speaker data.

Purpose of the Study:

• To develop a fast and high-quality speech synthesis approach with minimal target speaker recordings.
• To reduce the substantial training costs associated with large auxiliary corpora in TTS.

Main Methods:

• Proposed a Statistical-based Compression Auxiliary Corpus (SCAC) algorithm to optimize auxiliary data.
• Developed a non-autoregressive model, CMDF-TTS, incorporating multi-level prosody and Denoising Diffusion Probabilistic Models (DDPMs).
• Utilized Conditional Variational Auto-Encoder Generative Adversarial Networks (CVAE-GAN) for further quality enhancement.

Main Results:

• The SCAC algorithm significantly improves model training speed without compromising speech naturalness.
• CMDF-TTS, enhanced by SCAC, demonstrates a balance between training efficiency and synthesized speech quality.
• Experimental results show superior performance compared to state-of-the-art models on Mandarin and English datasets.

Conclusions:

• The proposed SCAC algorithm effectively reduces auxiliary corpus requirements for TTS.
• CMDF-TTS offers a promising solution for efficient and high-quality speech synthesis.
• The approach successfully integrates prosody modeling and diffusion models for enhanced speech generation.