Related Experiment Videos
Auditory-visual speech perception examined by fMRI and PET.
Kaoru Sekiyama1, Iwao Kanno, Shuichi Miura
1Neuroscience Research Institute, National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan. sekiyama@fun.ac.jp
Neuroscience Research
|October 22, 2003
Summary
The McGurk effect reveals how visual speech strongly influences hearing when auditory input is unclear. Brain imaging shows the superior temporal sulcus (STS) is key for binding auditory and visual speech signals.
Area of Science:
- Neuroscience
- Cognitive Science
- Speech Perception
Background:
- Cross-modal binding integrates sensory information, crucial for speech perception.
- The McGurk effect demonstrates visual speech influencing auditory perception.
Purpose of the Study:
- Investigate neural mechanisms of cross-modal binding in auditory-visual speech.
- Examine the role of the superior temporal sulcus (STS) in integrating speech signals.
Main Methods:
- Functional magnetic resonance imaging (fMRI) and positron emission tomography (PET).
- Presented auditory, visual, and audiovisual McGurk effect stimuli.
- Manipulated auditory intelligibility (High vs. Low signal-to-noise ratio).
Main Results:
- Visual influence on speech perception intensified under low auditory intelligibility.
- Distinct brain regions activated for unimodal auditory (temporal cortex) and visual (MT/V5) stimuli.
- Audiovisual stimuli showed extended activation in the left temporal cortex and posterior STS, particularly in low intelligibility conditions.
Conclusions:
- The posterior left STS is implicated in cross-modal binding of auditory-visual speech.
- Auditory intelligibility modulates the neural processing of audiovisual speech integration.
- Brain activity patterns reflect the degree of visual influence on auditory speech perception.