Collaborative AI Dysarthric Speech Recognition System With Data Augmentation Using Generative Adversarial Neural...

Area of Science:

• Speech Recognition
• Artificial Intelligence
• Biomedical Engineering

Background:

• Dysarthric speech poses significant challenges for automatic speech recognition (ASR) systems.
• Existing ASR systems often struggle with the variability and reduced intelligibility of dysarthric speech.
• Enhancing ASR robustness for dysarthric speech is crucial for improving communication accessibility.

Purpose of the Study:

• To develop a novel collaborative system for converting dysarthric speech into non-dysarthric speech.
• To enhance the robustness and accuracy of ASR systems fine-tuned for dysarthric speech.
• To improve the naturalness and intelligibility of communication for individuals with dysarthria.

Main Methods:

• A three-stage data augmentation framework was employed, including synthetic speech generation (Tacotron2), tempo perturbation, and CycleGAN-based conversion with Inception-ResNet and temporal masking.
• The system focused on mapping phonological features while preserving speech structure and resolving temporal irregularities.
• Experiments were conducted using the UASpeech corpus to evaluate the system's performance.

Main Results:

• The proposed system demonstrated a significant reduction in word error rate (WER) compared to baseline approaches.
• The three-stage data enhancement reduced WER by 9.81% for Wav2Vec2-XLSR and 6.56% for Whisper-Tiny models.
• The best performing system achieved an average WER of 13.58%, indicating improved accuracy and naturalness.

Conclusions:

• The novel collaborative framework effectively improves the accuracy and naturalness of speech recognition for dysarthric speech.
• The three-stage data augmentation strategy is a key contributor to the system's success.
• This system offers a promising solution for more intelligible communication for individuals with dysarthria.