Contributions of local speech encoding and functional connectivity to audio-visual speech perception

Area of Science:

• Neuroscience
• Auditory and Speech Processing
• Multisensory Integration

Background:

• Visual cues, such as seeing a speaker's face, significantly enhance speech intelligibility, especially in noisy environments.
• Understanding the neural mechanisms underlying this multisensory benefit is crucial for comprehending speech perception.
• Previous research has explored auditory processing, but the network dynamics involving visual speech information remain less understood.

Purpose of the Study:

• To investigate the brain network mechanisms that support the enhancement of speech intelligibility by visual facial cues.
• To quantify local speech representations and directed brain connectivity using magnetoencephalography (MEG) data.
• To determine how acoustic signal-to-noise ratio (SNR) and visual context influence neural activity and connectivity during speech perception.

Main Methods:

• Acquired MEG data from human participants listening to speech under varying acoustic SNR and visual conditions.
• Quantified local speech representations by analyzing temporally entrained brain activity.
• Assessed directed functional connectivity along auditory and frontoparietal pathways.

Main Results:

• Speech encoding shifted from temporal and inferior frontal cortex (high SNR) to premotor and superior frontal cortex (low SNR).
• Changes in local encoding were associated with altered directed connectivity in the ventral stream and auditory-premotor axis.
• The behavioral advantage of visual speech was linked to increased functional connectivity between temporal and inferior frontal cortex, not local encoding changes.

Conclusions:

• Auditory-frontal interactions play a significant role in processing visual speech information.
• Enhanced functional connectivity within the ventral pathway facilitates speech comprehension in multisensory environments.
• These findings highlight the dynamic neural network adaptations supporting robust speech perception under challenging listening conditions.