Improving Electrolaryngeal Speech Enhancement via a Representation Learning Method Based on Integrated Text and...

Area of Science:

• Speech processing
• Artificial intelligence
• Biomedical engineering

Background:

• Electrolaryngeal (EL) speech, used by laryngectomees, lacks naturalness and intelligibility due to phonetic and temporal deviations from normal speech.
• Existing EL-speech-to-normal-speech conversion (EL2SP) methods using sequence-to-sequence voice conversion (VC) struggle with feature mismatches, leading to accumulated errors and limited performance.

Purpose of the Study:

• To develop a novel representation learning method for EL2SP that enhances mapping accuracy by integrating paired EL speech and text representations.
• To improve the naturalness and intelligibility of EL speech for better communication.

Main Methods:

• Proposed a novel representation learning method integrating paired electrolaryngeal speech and text representations.
• Incorporated text information to enrich intermediate representations and guide alignment learning between speech encoder and decoder.
• Applied autoencoder-style reconstruction training to inherit integrated representations without increasing model complexity.

Main Results:

• The proposed method, combined with data augmentation, outperformed baseline methods that solely learn speech representations.
• Achieved a 19.2% improvement in character error rate compared to a pretraining-fine-tuning baseline.
• Demonstrated a 0.67 increase in naturalness score, verifying the effectiveness of the integrated representation approach.

Conclusions:

• Integrating text information into representation learning significantly enhances EL-speech-to-normal-speech conversion.
• The proposed method offers a more effective approach to improving EL speech quality for laryngectomees.
• This technique holds promise for advancing assistive communication technologies for individuals with speech impairments.