Enhancing Dysarthric Voice Conversion with Fuzzy Expectation Maximization in Diffusion Models for Phoneme Prediction

Area of Science:

• Speech Processing
• Artificial Intelligence
• Assistive Technologies

Background:

• Dysarthria, a motor speech disorder, impairs speech intelligibility due to neurological damage.
• Existing voice conversion (VC) systems struggle with the variability of dysarthric speech, particularly in phoneme prediction.
• Accurate phoneme prediction is crucial for improving dysarthric VC quality and communication.

Purpose of the Study:

• To develop a novel approach for enhanced phoneme prediction in dysarthric speech.
• To improve the quality and intelligibility of voice conversion for individuals with dysarthria.
• To integrate Fuzzy Expectation Maximization (FEM) with diffusion models for robust speech processing.

Main Methods:

• Combined Fuzzy Expectation Maximization (FEM) clustering with Diffusion Probabilistic Models (DPM).
• Utilized diffusion models for noise simulation to enhance speech signal robustness.
• Employed FEM for iterative phoneme boundary optimization to reduce uncertainty.
• Trained the system on the Saarland University Voice Disorder dataset, processing speech in the Mel-spectrogram domain.

Main Results:

• Significantly improved phoneme prediction accuracy and overall voice conversion quality.
• Achieved higher Mean Opinion Scores (MOS) for naturalness, intelligibility, and speaker similarity compared to StarGAN-VC and CycleGAN-VC.
• Demonstrated lower Word Error Rate (WER) for both mild and severe dysarthria, indicating enhanced speech intelligibility.

Conclusions:

• The integration of FEM and diffusion models substantially improves handling dysarthric speech irregularities.
• The method shows robustness, maintaining speech naturalness and intelligibility without a speaker-encoder.
• This approach offers a promising foundation for developing reliable assistive communication technologies and personalized speech therapy for dysarthria.