Electrocorticography reveals continuous auditory and visual speech tracking in temporal and occipital cortex

Area of Science:

• Neuroscience
• Auditory Perception
• Visual Perception

Background:

• Natural speech perception involves integrating continuous auditory and visual signals.
• Previous research often used simplified stimuli like single words or syllables.
• Understanding continuous speech processing is crucial for real-world communication.

Purpose of the Study:

• To investigate neural mechanisms of continuous audiovisual speech processing.
• To identify brain regions involved in tracking auditory and visual speech dynamics.
• To explore the role of visual speech in enhancing auditory speech perception.

Main Methods:

• Electrocorticography (ECoG) was used in epileptic patients with subdural electrodes.
• Participants listened to and watched continuous audiovisual speech (sentences).
• Partial correlation analysis was employed to examine neural tracking of speech signals.

Main Results:

• Posterior superior temporal gyrus (pSTG) and medial occipital cortex tracked auditory and visual speech envelopes.
• These regions, plus inferior temporal cortex, showed stronger responses to dynamic talking faces.
• Occipital cortex and pSTG encode temporal information from both auditory and visual speech dynamics.

Conclusions:

• pSTG and occipital cortex are critical for processing continuous audiovisual speech.
• Dynamic visual speech significantly influences neural processing compared to static visual cues.
• Visual speech tracking in pSTG may enhance the perception of challenging auditory speech.