Advanced accent/dialect identification and accentedness assessment with multi-embedding models and automatic speech...

Area of Science:

• Speech processing
• Computational linguistics
• Artificial intelligence

Background:

• Accurate accent classification and accentedness assessment are challenging due to diverse speech variations.
• Existing methods struggle with the complexity of accents and dialects in non-native speakers.

Purpose of the Study:

• To enhance accent classification and non-native accentedness assessment using pretrained language identification (LID) and speaker identification (SID) models.
• To develop a multi-embedding system for superior accent identification (AID) accuracy.
• To investigate the use of automatic speech recognition (ASR) and AID models for objective accentedness estimation.

Main Methods:

• Leveraging embeddings from advanced pretrained LID and SID models.
• Integrating LID and SID embeddings with an end-to-end (E2E) AID model.
• Utilizing an E2E ASR model trained on American English (en-US) and an AID model's en-US output for scoring.
• Correlating objective scores with subjective human perception scores.

Main Results:

• Pretrained LID and SID models effectively encode accent and dialect information.
• A multi-embedding AID system incorporating LID, SID, and E2E AID embeddings achieves superior accuracy.
• ASR error rate and AID model output provide reliable objective accentedness scores.
• Objective scores show strong correlation with each other and with subjective human assessments.

Conclusions:

• Pretrained LID and SID models significantly improve accent classification and accentedness assessment.
• The proposed multi-embedding AID system offers enhanced accuracy for accent identification.
• ASR and AID-based systems provide a reliable and valid method for objective accentedness estimation.
• These advancements have significant implications for language learning, speech intelligibility, and speaker recognition technologies.