Improving Electrolaryngeal Speech Enhancement via a Representation Learning Method Based on Integrated Text and...

Laryngectomees use an electrolarynx to produce electrolaryngeal (EL) speech for communication. However, EL speech lacks essential phonetic features and exhibits a temporal structure that significantly deviates from normal speech, leading to poor naturalness and intelligibility. EL-speech-to-normal-speech conversion (EL2SP) based on sequence-to-sequence (seq2seq) voice conversion (VC) is promising to address this deficiency; however, due to huge mismatches between EL and normal speech features, seq2seq-based mapping inevitably accumulates errors, limiting conversion performance. To address this issue, we propose a novel representation learning method that integrates paired EL speech and text representations, facilitating more effective mapping to target speech. Specifically, we incorporate text information to enrich intermediate representations and guide alignment learning between speech encoder and decoder. Next, we apply autoencoder-style reconstruction training (only updating the encoder), enabling final EL2SP model to inherit integrated representations without increasing complexity. Experimental results demonstrate that our method, combined with data augmentation, outperforms the baselines that solely learn speech representations. Notably, our approach achieves a 19.2% improvement in character error rate and a 0.67 increase in naturalness score compared to a typical pretraining-fine-tuning baseline, verifying its effectiveness.