Related Experiment Videos
Multimodal contribution to speech perception revealed by independent component analysis: a single-sweep EEG case
D E Callan1, A M Callan, C Kroos
1ATR-I Brain Activity Imaging Center, ATR Human Information Processing Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, 619-0288, Kyoto, Japan. dcallan@hip.atr.co.jp
Brain Research. Cognitive Brain Research
|February 13, 2001
Summary
Visualizing facial motion improves speech intelligibility in noise by increasing high-frequency brain activity. This multimodal integration effect involves the superior temporal gyrus and other cortical areas.
Area of Science:
- Neuroscience
- Auditory Perception
- Multimodal Integration
Background:
- Speech intelligibility in noisy environments is a significant challenge.
- Visual speech cues, such as facial movements, can enhance auditory speech perception.
- The neural mechanisms underlying this audiovisual speech enhancement are not fully understood.
Purpose of the Study:
- To investigate the electroencephalographic (EEG) correlates of speech intelligibility enhancement.
- To explore the role of facial motion in audiovisual speech perception using independent component analysis (ICA).
- To identify brain regions involved in integrating visual speech information with auditory signals.
Main Methods:
- Single-sweep electroencephalographic (EEG) data acquisition.
- Independent Component Analysis (ICA) to isolate neural sources.
- Wavelet analysis of independent component (IC) activation waveforms.
- Current source density (CSD) analysis for source localization.
Main Results:
- Increased high-frequency energy was observed in two ICs associated with the visual enhancement effect.
- CSD analysis localized one IC's activity primarily to the superior temporal gyrus.
- Another IC showed distributed activity across multiple cortical areas.
Conclusions:
- Visual facial motion enhances speech intelligibility by modulating high-frequency neural activity.
- The superior temporal gyrus plays a role in multimodal speech integration.
- Widespread cortical networks may underlie the global mapping of visual speech enhancement.