Perceptual evaluation of substitution voices: development and evaluation of the (I)INFVo rating scale

Substitution voicing cannot be evaluated accurately by the GRBAS perceptual rating scale, and there is a need for a valuable alternative. Therefore, we developed and tried out a perceptual rating scale, consisting of five new parameters: impression, intelligibility, noise, fluency and voicing, each to be scored between 0 (very bad score) to 10 (very good score for a substitution voice). In analogy to the GRBAS scale, they are then converted to deviance scores ranging from 0 (similar to good substitution voicing) to 3 (very deviant from good substitution voicing). Inter-individual agreement measured in a set of 24 semi-professional jury members seemed to be moderate for all parameters. Mean figures of 0.52, 0.51, 0.46, 0.53 and 0.46 are obtained for the parameters impression, intelligibility, noise, fluency and voicing, respectively. Because a high correlation exists between the first two parameters (0.917) and relying on the correlation figures between the two "I"s and the other parameters (correlation values for "impression" vary from 0.79-0.86; values for "intelligibility" range from 0.74-0.83), we suggest to discard the parameter impression, which turns the actual IINFVo scale into INFVo. The proposed (I)INFVo perceptual rating scale seems promising for the assessment of substitution voicing. Eventual improvements and practical proposals are discussed.