Seeing and hearing others and oneself talk.

Area of Science:

• Neuroscience
• Auditory Perception
• Speech Science

Background:

• Auditory perception is primarily studied in isolation.
• The interplay between speech production and perception remains an active area of research.
• The influence of visual and motor speech cues on auditory processing requires further investigation.

Purpose of the Study:

• To investigate how visual and motor aspects of speech influence auditory perception.
• To quantify the effects of observing articulation (others' and self) and silent articulation on speech sound identification.
• To explore the neural underpinnings of these cross-modal interactions.

Main Methods:

• Twenty subjects participated in three experimental conditions.
• Stimuli included acoustic speech sounds paired with visual articulatory gestures (discordant and concordant).
• Participants' auditory perception was assessed under conditions of observing others' articulation, observing their own silent articulation, and silent articulation without visual feedback.

Main Results:

• Observing discordant articulatory gestures significantly impaired speech sound identification, eliciting a strong McGurk effect.
• Similar effects were observed when subjects viewed their own silent articulation.
• Silent articulation alone, without visual feedback, also produced a smaller but significant effect.
• Concordant articulation, both observed and self-generated, improved speech sound identification.

Conclusions:

• Visual and motor speech processes significantly modulate auditory perception.
• These cross-modal influences are suggested to arise from altered activity within the auditory cortex.
• The findings strongly support a close, integrated relationship between speech perception and speech production.