Percent Consonant Correct as an Outcome Measure for Cleft Speech in an Intervention Study

Percent consonant correct (PCC) was originally described by Shriberg and Kwiatkowski [J Speech Hear Disord. 1982 Aug;47(3):256-70] as a severity metric for phonological speech disorders, and has been adapted and used in many studies on speech sound disorders. It is well-recognized that cleft speech is complex, consisting of several interacting parameters assessed simultaneously, with error sounds not in the listener's own language. In speech outcome studies, narrow phonetic transcription and the reporting of intra- and inter-rater reliability are acknowledged as the gold standard. However, cleft speech brings special challenges to this task, as complex speech disorders are known to be associated with low transcriber agreement. Recent studies informed the decision to use PCC as the primary outcome measure in a cleft speech intervention study, given its common usage and familiarity. The aim was to specifically evaluate the intra- and inter-rater reliability of PCC in an intervention study, in contrast to other types of speech outcome studies.

Two trained and experienced listeners analyzed 119 recordings, randomly selected from five data points before, during, and following intervention. The PCC score was separately calculated for words and sentences/phrases.

Using intraclass correlations (ICCs), Phase 1 results showed poor reliability for targets elicited for words (ICC = 0.07) and sentences/phrases (ICC = 0.42). Differences in classification of errors as glottal stops and consonant deletion accounted for this. Following further training, a second reliability study was undertaken showing improvement in the number of targets elicited in words (ICC = 0.85) and sentences/phrases (ICC = 0.94). There was very good inter-rater reliability for the PCC score on the word dataset (ICC = 0.9) and the sentence dataset (ICC = 0.88). Very good intra-rater reliability (ICC = 1.0) was found for the PCC score in both words and sentences/phrases for each listener. One listener consistently gave higher modified PCC scores.

In cleft speech intervention studies, reliability of the number of targets elicited should be reported. Listeners need to distinguish between glottal articulation and consonant deletion, in order that the PCC score is meaningful. Attention should be paid to where listeners are reliable, but their pattern of scores consistently differs but in a consistent way. More research is needed on measuring the resolution of articulation difficulties in cleft intervention studies.