Emphasizing unseen words: New vocabulary acquisition for end-to-end speech recognition

Area of Science:

• Speech Recognition
• Natural Language Processing
• Machine Learning

Background:

• Automatic speech recognition (ASR) systems struggle with new and evolving vocabulary, known as out-of-vocabulary (OOV) words.
• Traditional methods require extensive retraining, posing challenges for adapting to dynamic language.
• Existing research often focuses on post-processing, neglecting acoustic modeling biases.

Purpose of the Study:

• To improve the acoustic recognition of OOV words in ASR systems.
• To develop methods that address OOV word challenges at an earlier processing stage.
• To enable continuous learning and adaptation of ASR systems to new vocabulary.

Main Methods:

• Generating OOV words using text-to-speech (TTS) systems to create synthetic training data.
• Implementing loss rescaling techniques (sentence-level and word-level) to focus neural networks on OOV words.
• Combining loss rescaling with regularization methods like L2 and Elastic Weight Consolidation (EWC) to prevent catastrophic forgetting.

Main Results:

• The proposed loss rescaling method significantly improves recall rates for OOV words.
• A slight decrease in word error rate is observed, indicating maintained overall performance.
• Word-level rescaling demonstrates greater stability and higher precision and recall compared to utterance-level rescaling.
• Combined loss rescaling and weight consolidation facilitate continual learning in ASR systems.

Conclusions:

• Loss rescaling offers an effective approach to enhance OOV word recognition in ASR systems.
• The methods presented allow ASR systems to acoustically recognize OOV words more effectively.
• The study supports the continual learning of ASR systems, enabling them to adapt to new vocabulary over time.