The Post-Stroke Speech Transcription (PSST) Challenge.

Area of Science:

• Computational linguistics
• Speech processing
• Neuroscience of language

Background:

• Aphasia, a language disorder post-stroke, significantly impacts speech production.
• Accurate speech transcription is crucial for assessing and rehabilitating aphasia.
• Existing speech recognition technologies often struggle with disordered speech patterns.

Purpose of the Study:

• To introduce the Post-Stroke Speech Transcription (PSST) challenge and its associated dataset.
• To evaluate the performance of automatic speech recognition (ASR) systems on phonemic transcription of post-stroke speech.
• To assess the utility of improved phonetic transcripts in a downstream task of identifying correct word pronunciation.

Main Methods:

• Creation of a novel dataset from AphasiaBank, including audio recordings and phonemic transcripts for confrontation naming tasks.
• Task A: Development and evaluation of ASR models for phonemic transcription using Phoneme Error Rate (PER) and Feature Error Rate (FER).
• Task B: Evaluation of a classification algorithm to determine correct pronunciation of target words, utilizing baseline and enhanced transcripts.

Main Results:

• The leading ASR model achieved 9.9% FER and 20.0% PER, representing significant improvements over the baseline.
• Challengers did not outperform the baseline algorithm in the direct pronunciation assessment task (Task B).
• Integrating Task A's improved transcripts into the Task B algorithm yielded a 92.8% accuracy and 0.921 F1 score.

Conclusions:

• The PSST challenge dataset and tasks facilitate advancements in ASR for disordered speech.
• Improved phonemic transcription significantly enhances the accuracy of downstream speech assessment tasks.
• Further research can leverage these findings for better clinical tools for aphasia.