Auditory, visual and audiovisual perception of segmental speech features by severely hearing-impaired children

Auditory alone, visual alone and audiovisual recognition of consonant-vowel consonant syllables were measured in 32 severely hearing-impaired children with hearing loss (PTA) in a narrow range around 90 dB HL when using their hearing aids. Multidimensional scaling analysis (INDSCAL) and information transmission analysis (ITA), applied to the confusion matrices obtained from the responses in each presentation mode and for each phoneme category, revealed perceptual dimensions and percentages of transmitted feature information (PTI). These were studied in relation to PTA, the auditory alone score and in relation to the efficiency of the audiovisual interaction (enhancement) over the probalistic summation of the auditory alone and visual alone score. INDSCAL analysis shows that auditory alone recognition of vowels is based on the perceptual dimensions F2 and F1 and that of consonants on the dimensions 'frication' and 'voicing'. In the auditory mode the interpretation of the INDSCAL dimensions in the stimulus spaces is in reasonable agreement with the ITA results. PTI decreases gradually with decreasing auditory alone phoneme score. Audiovisual recognition of vowels is based on a combination of the auditory dimension 'open/closed' (F1), and the visual dimensions 'lip rounding' and 'vertical lip opening'. Audiovisual recognition of initial consonants is based on a combination of the visual dimension 'front/back' and the auditory dimension 'continuance'. Recognition of final consonants is based on a combination of the visual dimension 'front/back' and an uninterpretable dimension. The perceptual dimensions are independent of both the level of the auditory alone phoneme score and audiovisual enhancement. Audiovisual enhancement is mainly a property of an individual and independent of both auditory alone and visual alone scores. ITA analysis, based on a phonological classification of the features, supports the results of the INDSCAL analysis in the auditory alone mode. It is not useful in the description of the audiovisual interaction, probably due to the phonological basis of the feature classification.