Mandarin Electrolaryngeal Speech Recognition Based on WaveNet-CTC

Area of Science:

• Speech Processing
• Artificial Intelligence
• Laryngology

Background:

• Electrolaryngeal (EL) speech presents challenges for laryngectomees due to its mechanical nature and limited intelligibility.
• Traditional methods struggle to effectively process and enhance EL speech.
• Improving communication for individuals with laryngeal cancer is a critical area of research.

Purpose of the Study:

• To enhance the intelligibility of Chinese Mandarin electrolaryngeal (EL) speech for laryngectomees.
• To develop an automatic speech recognition (ASR) system capable of converting EL speech into natural-sounding speech.
• To leverage deep learning models for improved speech recognition in challenging acoustic conditions.

Main Methods:

• Designed an ASR system utilizing the WaveNet deep learning model and connectionist temporal classification (CTC).
• The system incorporates acoustic, language, and decoding models, extracting acoustic features during preprocessing.
• Trained the ASR system on a dataset comprising 3,230 EL speech utterances and 10,000 healthy speech utterances.

Main Results:

• The proposed WaveNet-CTC based ASR system demonstrated superior stability and generalizability compared to traditional methods.
• The system achieved high accuracy in recognizing Chinese characters, words, and sentences of varying lengths.
• The highest ASR accuracy reached 83.24%, indicating effective recognition of EL speech, despite phoneme confusion in certain consonants.

Conclusions:

• Electrolaryngeal (EL) speech can be effectively recognized and converted into healthy speech using an ASR system based on WaveNet-CTC.
• The developed method offers enhanced generalization performance and stability over conventional approaches.
• This technology holds significant potential for improving communication for laryngectomees.