End-to-End Automatic Pronunciation Error Detection Based on Improved Hybrid CTC/Attention Architecture

Area of Science:

• Speech Technology
• Artificial Intelligence
• Computational Linguistics

Background:

• Traditional automatic pronunciation error detection (APED) relies on complex automatic speech recognition (ASR) systems.
• Advancements in deep learning have led to mature end-to-end ASR technologies, offering new possibilities for APED.
• Existing APED methods often require force alignment and segmentation, increasing complexity.

Purpose of the Study:

• To develop a novel, simplified APED algorithm leveraging end-to-end ASR.
• To improve the performance and applicability of APED for computer-assisted pronunciation training (CAPT).
• To create an L1-independent solution for pronunciation error detection.

Main Methods:

• Constructed an end-to-end ASR system using a hybrid Connectionist Temporal Classification (CTC) and attention (seq2seq) architecture.
• Incorporated an adaptive parameter to enhance the synergy between CTC and attention models.
• Applied the improved ASR system to the Mandarin APED task.

Main Results:

• The hybrid CTC/attention ASR system achieved good results in Mandarin APED.
• The proposed APED method eliminates the need for force alignment, segmentation, and multiple complex models (acoustic, language).
• The system demonstrates accuracy comparable to state-of-the-art DNN-DNN ASR systems and superior F-measure performance for APED.

Conclusions:

• The developed end-to-end ASR-based APED system offers a convenient and straightforward solution.
• This approach is suitable for L1-independent computer-assisted pronunciation training (CAPT).
• The hybrid CTC/attention architecture provides a strong foundation for effective pronunciation error detection.