Tri-word presentations with phonemic scoring for practical high-reliability speech recognition assessment

Speech recognition test reliability is optimized with 450 test items, and the Computer Assisted Speech Recognition Assessment (CASRA) test is a practical approach for achieving this goal by combining 50 presentations of 3 consonant-vowel nucleus-consonant (CNC) words each with phonemic scoring (S. A. Gelfand, 1998). However, optimized reliability might not be essential if reliability is as high as possible in light of practical constraints and what the clinician is trying to do with the results. The CASRA paradigm addresses these compromise goals with a reduced number of 3-word sets: 25 sets yield 25 (groups) x 3 (words) x 3 (phonemes) = 225 test items, 20 sets give 20 x 3 x 3 = 180 items, and 10 sets provide 10 x 3 x 3 = 90 items. This study addressed the empirical reliability of such an approach, and the extent to which results on shortened versions predict full-test scores. Test and retest scores were obtained for 10-, 20-, and 25-set versions of the CASRA for 144 participants with a wide range of hearing ability. For group data, first and second scores were highly correlated and not significantly different from each other for all 3 test sizes. Performance based on 20 and 25 sets accounted for roughly 97% of the variance of full (50-set) test scores, and scores based on 10 sets accounted for about 88% of the full-test variance. Individual test-retest reliability agreed with theoretical expectations based on 95% binomial confidence intervals. Cases outside the 95% confidence limits were 7.6% for 10 sets, and 3.5% for 20 and 25 sets with phoneme scoring, and 4.9% for 10 and 20 sets and 3.5% for 25 sets with word scoring. The shortened CASRA is a practical way to achieve improvements in reliability over traditional word tests. The 20-set version may approximate the strongest compromise when trying to shorten test size without appreciably reducing reliability for clinical purposes. However, the 10-set version is probably a more practical approach for routine use because it accounts for 88% of full-test variance, is more reliable than a traditional 75-word test, and does not appear to be subject to significant short-term learning effects.