STYLETTS-VC: ONE-SHOT VOICE CONVERSION BY KNOWLEDGE TRANSFER FROM STYLE-BASED TTS MODELS

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• One-shot voice conversion (VC) enables speech style transfer using minimal target speaker data.
• Disentangling speaker identity from speech content is a key challenge in VC.
• Existing methods often require extensive training data or struggle with naturalness.

Purpose of the Study:

• To develop a novel approach for one-shot voice conversion (VC).
• To achieve disentangled speech representation for high-fidelity voice transfer.
• To improve the naturalness and similarity of converted speech.

Main Methods:

• Utilizing transfer learning from style-based text-to-speech (TTS) models.
• Employing cycle-consistent and adversarial training for transcription-guided VC.
• Implementing a teacher-student knowledge transfer with a mel-spectrogram encoder and data augmentation for text-independent representation.

Main Results:

• The proposed method successfully disentangles speaker identity and speech content.
• Achieved high fidelity and similarity in one-shot voice conversion.
• Significantly outperformed state-of-the-art models in subjective evaluations of naturalness and similarity.

Conclusions:

• The novel approach effectively addresses the challenge of disentangled speech representation in one-shot VC.
• Transfer learning from TTS models offers a promising direction for advanced voice conversion.
• The text-independent method enhances the practicality and performance of one-shot VC systems.