Dysarthric Speech Transformer: A Sequence-to-Sequence Dysarthric Speech Recognition System.

Area of Science:

• Speech and Language Processing
• Biomedical Engineering
• Artificial Intelligence

Background:

• Dysarthria significantly impairs speech intelligibility, creating a critical need for effective assistive technologies.
• Current automatic speech recognition (ASR) systems perform poorly on dysarthric speech, especially in severe cases.
• Advanced transformer-based ASR models show promise but are underexplored for dysarthric speech due to data limitations.

Purpose of the Study:

• To develop and evaluate a novel transformer-based ASR system specifically designed for dysarthric speech.
• To address the challenge of limited training data for dysarthric speakers.
• To improve the accuracy and usability of ASR technology for individuals with speech impairments.

Main Methods:

• Proposed a customized deep transformer architecture named Dysarthric Speech Transformer.
• Implemented a two-phase transfer learning pipeline using healthy speech data.
• Employed audio data augmentation techniques and investigated neural freezing configurations.
• Trained 45 speaker-adaptive dysarthric ASR models.

Main Results:

• The transfer learning pipeline and data augmentation significantly improved ASR performance.
• Deeper transformer architectures were found to be crucial for enhanced accuracy.
• The proposed ASR system outperformed state-of-the-art methods for 73% of dysarthric subjects.
• Achieved up to a 23% improvement in accuracy for certain individuals.

Conclusions:

• The Dysarthric Speech Transformer effectively addresses the limitations of current ASR systems for dysarthric speech.
• Transfer learning and data augmentation are vital strategies for overcoming data scarcity in dysarthric ASR.
• The developed system offers a promising advancement in assistive technology for individuals with speech impairments.