Clinical assessment and interpretation of dysarthria in ALS using attention based deep learning AI models

Speech dysarthria is a key symptom of neurological conditions like ALS, yet existing AI models designed to analyze it from audio signal rely on handcrafted features with limited inference performance. Deep learning approaches improve accuracy but lack interpretability. We propose an attention-based deep learning AI model to assess dysarthria severity based on listener effort ratings. Using 2,102 recordings from 125 participants, rated by three speech-language pathologists on a 100-point scale, we trained models directly from recordings collected remotely. Our best model achieved R² of 0.92 and RMSE of 6.78. Attention-based interpretability identified key phonemes, such as vowel sounds influenced by 'r' (e.g., "car," "more"), and isolated inspiration sounds as markers of speech deterioration. This model enhances precision in dysarthria assessment while maintaining clinical interpretability. By improving sensitivity to subtle speech changes, it offers a valuable tool for research and patient care in ALS and other neurological disorders.