Confusion2Vec 2.0: Enriching ambiguous spoken language representations with subwords

Area of Science:

• Natural Language Processing
• Speech Recognition
• Machine Learning

Background:

• Word vector representations are crucial for spoken language understanding.
• Existing methods often overlook ambiguities inherent in human speech.
• Automatic Speech Recognition (ASR) systems produce lattice outputs that contain rich information about speech ambiguities.

Purpose of the Study:

• To propose a novel word vector space estimation method called Confusion2vec.
• To encode semantic, syntactic, and acoustic ambiguity information into word vectors.
• To improve spoken language understanding, particularly in the presence of ASR errors.

Main Methods:

• Unsupervised learning on lattices from ASR systems.
• Encoding words using constituent subword character n-grams.
• Evaluating word vectors on analogy, word similarity, and spoken language intent detection tasks.

Main Results:

• Confusion2vec effectively represents acoustic perceptual ambiguities using subword encoding.
• The method significantly outperforms existing word vector representations on erroneous ASR outputs.
• Achieved up to 13.12% improvement in intent detection on the ATIS dataset.

Conclusions:

• Confusion2vec provides a robust spoken language representation by incorporating human language ambiguities.
• Subword modeling is beneficial for representing acoustic ambiguities.
• Confusion2vec eliminates the need for retraining natural language understanding models on ASR transcripts.