How Reliable Is the Auditory-Perceptual Evaluation of Phonation Onset Hardness?

Generally, phonation onset hardness has been similarly defined and categorized throughout literature. There are typically three categories: soft, breathy, and hard onset. Phonation onset hardness is relevant in both assessment and treatment of various voice disorders and is usually determined on an auditory-perceptual basis. However, as far as we know, reliability of clinical auditory-perceptual indications of phonation onset hardness has not been investigated yet. Therefore, this study aimed at evaluating intrarater as well as interrater reliability of auditory-perceptual assessment of phonation onset hardness by speech and language pathologists.

After deidentification, the sentence-initial word [e.rst] was extracted from read text recordings of 20 subjects with various voice disorders and complaints. These 20 samples were purposefully selected to represent as much as possible the whole phonation onset hardness continuum. The auditory-perceptual rating protocol consisted of the paired comparison paradigm, in which all listeners were asked to perceptually compare phonation onset hardness of every fragment with the other 19 samples. This resulted in a ranking of the 20 samples, from hardest to softest phonation onset. Four speech and language pathologists agreed to compare phonation onset hardness according to this paradigm. The single-measures intraclass correlation coefficient (ICC) for absolute agreement was applied to determine the degree of reliability within as well as between raters.

Intrarater ICC's showed acceptable reliability for two raters, yet poor reliability for the other two raters. Interrater ICC's demonstrated low reliability in general. Zooming in on the ratings shows that three of four evaluators agreed on which sample had the softest phonation onset, and that only two raters agreed on which recording had the hardest phonation onset.

It can be concluded that there is considerable variability within and across raters when asked to indicate the word with the hardest phonation onset. This is the first study to approach auditory interpretation of phonation onset hardness with the paired comparison task. Questions are raised about the clinical utility of phonation onset hardness perception, the potential role of training, and the importance of a more objective yet clinically feasible measurement tool.