An artificial intelligence-based audio recognition system for dysarthria home-based rehabilitation: A...

Dysarthria is a common motor speech disorder affecting individuals with neurological conditions, with high prevalence among stroke survivors. Effective treatment of dysarthria requires accurate assessment of severity and the provision of speech exercises tailored to the degree of impairment. However, traditional speech intervention faces several limitations, including limited accessibility to therapists and low treatment adherence rates. AI-driven solutions have the potential to address these limitations by automating the detection of dysarthria severity and recommending personalized exercises. Although substantial progress has been made in the automated detection and classification of dysarthria severity, the performance of these approaches on less controlled, real-world speech remains insufficiently characterized. This proof-of-concept study describes the development and evaluation of Whisper-ST1, a novel framework for classifying dysarthric speech severity from audio files by fine-tuning a pretrained automatic speech recognition model. Our framework builds on previous studies by incorporating real-world data alongside traditional research datasets, adapting a model architecture pretrained on a large corpus of audio data, applying a progressive unfreezing strategy to prevent catastrophic forgetting of relevant speech features while promoting flexible adaptation to the new task, and implementing an ordinal regression loss to minimize the impact of misclassification errors. Ablation experiments demonstrate the performance contributions of individual pipeline components in isolation. Whisper-ST1 achieves 79% [67% - 100%] per-speaker F1 in the classification of dysarthria severity when tested on real-world samples, and shows strong agreement with two speech therapists' ratings of dysarthria (Weighted Cohen's kappa of 0.814 [0.400 - 1.000] and 0.878 [0.647 - 1.000]), comparable to inter-therapist agreement (0.851 [0.550 - 1.000]). Model errors are predominantly confined to adjacent severity levels, suggesting preservation of ordinal structure. These findings demonstrate the feasibility of leveraging pretrained speech models for dysarthria severity classification in heterogeneous audio conditions. and provide a foundational step toward AI-assisted tools for remote monitoring and support of dysarthria rehabilitation.