Reliability of the Dutch Cleft Speech Evaluation Test and Conversion to the Proposed Universal Scale

The Dutch cleft speech evaluation test (DCSET) has been implemented by the speech-language pathologists nationwide in the Netherlands since 2003, but the inter- and intrarater reliability was unknown. Two speech-language pathologists experienced in evaluating cleft speech assessed audio recordings of 20 children with cleft speech using the DCSET, and reassessed the recordings 2 weeks later. Intra- and interrater reliability was calculated, but found to be unacceptable after the first phase of this study using audio recordings. Following consensus training and some modifications in the scoring scales, the study was repeated using video recordings of 20 different children with cleft speech. Results from the second phase of this study using standardized video recordings showed fair, moderate, and good reliability on different subsets of the DCSET. Intrarater reliability (Kappa 0.59-1.00) was greater than interrater reliability (Kappa 0.33-0.79). Interrater reliability agreement was good (Kappa 0.63-0.79) for consonant production errors and speech understandability and acceptability, moderate (Kappa 0.59) for the resonance of the nasal passage, and fair (Kappa 0.33-0.37) for the resonance of the mixed and denasal passages. Subsequently, an algorithm was made to convert the DCSET scales to universal scales for international comparison of cleft speech as suggested by Henningsson et al in 2008.