Audiovisual perception of speech in noise and masked written text

The aim of this study was to examine the support obtained from degraded visual information in the comprehension of speech in noise.

We presented sentences auditorily (speech reception threshold test), visually (text reception threshold test), and audiovisually. Presenting speech in noise and masked written text enabled the quantification and systematic variation of the amount of information presented in both modalities. Eighteen persons with normal hearing (aged 19 to 31 yr) participated. For half of them a bar pattern masked the text and for the other half random dots masked the text. The text was presented simultaneously or delayed relative to the speech. Using an adaptive procedure, the amount of information required for a correct reproduction of 50% of the sentences was determined for both the unimodal and the audiovisual stimuli. Bimodal support was defined as the difference between the observed bimodal performance and that predicted by an independent channels model. Nonparametric tests were used to evaluate the bimodal support and the effect of delaying the text.

Masked text substantially supported the comprehension of speech in noise; the bimodal support ranged from 15% to 25% correct. A negative effect of delaying the text was observed in some conditions for the participants who were presented the text masked by the bar pattern.

The ability of participants to reproduce bimodally presented sentences exceeds the performance as predicted by an independent channels model. This indicates that a relatively small amount of visual information can substantially augment speech comprehension in noise, which supports the use of visual information to improve speech comprehension by participants with hearing impairment, even if the visual information is incomplete.