Convolution-Augmented Transformers for Enhanced Speaker-Independent Dysarthric Speech Recognition.

Area of Science:

• Speech and Language Processing
• Artificial Intelligence
• Biomedical Engineering

Background:

• Dysarthria is a motor speech disorder impacting verbal communication.
• Existing Automatic Speech Recognition (ASR) systems struggle with dysarthric speech due to data scarcity and speaker variability.
• Speaker-independent (SI) approaches are crucial for developing ASR systems that assist individuals with communication impairments.

Purpose of the Study:

• To develop and evaluate speaker-independent (SI) models for Automatic Speech Recognition (ASR) tailored to dysarthric speech.
• To address the challenge of data scarcity in dysarthric ASR using transfer learning and parameter-efficient fine-tuning (PEFT).
• To establish a benchmark framework for assessing the generalizability of SI models through cross-dataset validation.

Main Methods:

• Developed Conformer-based SI models using a three-stage transfer-learning pipeline with selective layer freezing PEFT.
• Pre-trained models on standard speech and progressively adapted them to two distinct dysarthric datasets (TORGO and UA-Speech).
• Implemented a cross-dataset validation strategy to evaluate model generalizability in a realistic cross-dataset scenario.

Main Results:

• The proposed dysarthric SI models outperformed all baseline systems on both isolated and continuous speech recognition tasks.
• Significant improvements were observed: 21.9% increase in word recognition accuracy for isolated speech and 18.5% reduction in word error rate for continuous speech on the TORGO dataset.
• On UA-Speech, the optimal SI model showed a 14.6% improvement over Whisper and 28.3% over the base model for isolated speech recognition.

Conclusions:

• The developed dysarthric SI models demonstrate superior performance in recognizing dysarthric speech, offering a promising solution for communication assistance.
• Cross-dataset validation revealed that models may transcribe isolated words instead of continuous speech for severe dysarthria, indicating a need for enhanced SI model generalization.
• Further research is required to improve the generalization capabilities of SI models for continuous dysarthric speech recognition.