Lexical mediation between sight and sound in speechreading.

Area of Science:

• Auditory perception
• Audiovisual speech processing
• Cross-modal interactions

Background:

• Understanding how visual speech information influences auditory speech perception is crucial.
• Previous research suggests visual input can affect auditory processing, but the mechanisms are debated.

Purpose of the Study:

• To investigate if simultaneous speech reading (lip-reading) affects speech detection in envelope-matched noise.
• To determine if audio-visual correspondence influences auditory perception and if this effect is mediated by lexical knowledge.

Main Methods:

• Two experiments were conducted using envelope-matched noise.
• Participants detected a disyllabic utterance while viewing a speaker producing matching or non-matching utterances.
• Stimuli included real words (Experiment 1) and non-word materials (Experiment 2).

Main Results:

• Speech detection was not improved by audio-visual matches, suggesting reliance on low-level auditory cues.
• A relative shift in bias was observed with word stimuli, where noise was perceived as more speech-like when visually matched.
• This bias shift was absent with non-word stimuli.

Conclusions:

• Simultaneous speech reading does not directly enhance speech detection in this context.
• The results suggest that the influence of visual speech on auditory perception is lexically mediated.
• The mapping from visual to auditory speech information involves lexical processing, even with non-arbitrary visual-auditory correspondences.