The use of perceptual methods by new clinicians for assessing voice quality

The purpose of this study was to determine the validity and reliability of using perceptual ratings for assessing voice quality in patients with vocal fold nodules or polyps. A 13-dimension perceptual rating system was modeled after systems currently in clinical use. To meet the criterion of 80% mean interjudge reliability, eight hours of training were required for four previously inexperienced listeners. Extended vowel phonations of patients and controls were then rated blindly by the same listeners. Interjudge reliability was greater than .90 for three dimensions judged in the pathological phonations, while intrajudge test-retest agreement was less than 75% on five dimensions. Validity was demonstrated with 100% correct assignment to group by computing a discriminant function employing all dimensions. Despite the extensive training procedures used, our reliability data were not comparable to those reported when highly experienced judges have been used, suggesting that the task of perceptually rating voice quality is difficult and requires extensive professional experience.