Reliability and agreement of ratings of ataxic dysarthric speech samples with varying intelligibility

Indices of interjudge reliability and inter- and intrajudge agreement were calculated from the ratings made by 15 experienced speech clinicians on five deviant speech dimensions with respect to 15 speakers with ataxic dysarthria. Speakers were chosen to cover a wide range of speech intelligibility (16-97%) as measured by the sentence intelligibility transcriptions of the Assessment of Intelligibility of Dysarthric Speech (Yorkston & Beukelman, 1981). Intraclass correlation coefficients derived from each judge on two occasions were above .6 for imprecise consonants, excess and equal stress, and harsh voice, but below .6 for distorted vowels and below .5 for irregular articulatory breakdown. The last dimension also had the lowest percent agreement for the interjudge and intrajudge comparisons. Poor speech dimension definition seems to be the most likely source of error on irregular articulatory breakdown. Judges agreed equally well in rating dysarthric speech across the range from low to high intelligibility.