Noise-robust voice conversion with domain adversarial training.

Area of Science:

• Artificial Intelligence
• Speech Processing
• Machine Learning

Background:

• Voice conversion (VC) has advanced significantly, achieving studio-quality results.
• Real-world applications face challenges due to environmental noise degrading speech quality and speaker similarity.

Purpose of the Study:

• To develop a noise-robust voice conversion framework for real-world applications.
• To maintain high speech quality and speaker similarity in noisy conditions.

Main Methods:

• Proposed a novel encoder-decoder based framework incorporating speaker and content encoders, a decoder, and domain adversarial neural networks.
• Integrated disentangling speaker/content representation with domain adversarial training to create noise-invariant representations.
• Ensured speaker and content representations from clean and noisy speech reside in the same space.

Main Results:

• The proposed method successfully synthesized clean converted speech under various noisy test scenarios.
• The framework demonstrated robustness against seen and unseen noise types during training.
• Significant improvements in both speech quality and speaker similarity were observed.

Conclusions:

• The developed noise-robust voice conversion framework effectively handles environmental noise.
• The approach enhances the practical applicability of voice conversion in real-world noisy environments.
• Future work may explore further improvements in noise suppression and conversion accuracy.