Auditory pattern perception in the profoundly hearing impaired and lipreading of Dutch phonemes

Successful rehabilitation of the profoundly hearing impaired by means of a speech-processing hearing aid requires integration of auditory and visual speech information. In two studies we investigated (1) which perceptual dimensions play a role in processing various auditory patterns by profoundly hearing-impaired subjects, and (2) which Dutch consonants and vowels can be identified by the average lipreader. One of the important cues in auditory pattern discrimination seems to be the presence of temporal fluctuations (beats) in the signal, resulting from two closely placed frequency components. However, this feature is confounded with the perception of loudness. A second cue used by some subjects is the presence of high-frequency peaks. In lipreading, at least three groups of consonants and three vowel groups may be distinguished; phonemes within a group cannot be discriminated from each other. Important features for both consonants and vowels are degree of lip opening and lip activity (movement or rounding). These results suggest how the auditory speech signal might be coded so as to provide supplementary information to speechreading.