A Visual Cortical Network for Deriving Phonological Information from Intelligible Lip Movements

Area of Science:

• Neuroscience
• Cognitive Science
• Auditory Perception

Background:

• Lip-reading relies on mapping visual lip movements to speech sounds.
• The visual and motor cortices are involved in tracking lip movements, but their role in visuo-phonological mapping is unclear.

Purpose of the Study:

• To investigate if the visual cortex maps absent acoustic speech signals from silent lip movements.
• To determine if this mapping occurs at the visual cortex level and is functionally relevant.

Main Methods:

• Examined cortical entrainment to unheard forward (intelligible) and backward (unintelligible) acoustic speech during silent lip movements.
• Measured top-down input to the visual cortex from motor and auditory regions.
• Correlated top-down modulation with entrainment strength.

Main Results:

• The visual cortex showed stronger entrainment to the forward acoustic speech envelope compared to the backward envelope.
• This forward-backward difference in occipital entrainment was absent for observed lip movements.
• Top-down input from premotor, motor, somatosensory, and temporal regions to the visual cortex correlated with entrainment to absent forward speech.

Conclusions:

• The visual cortex demonstrates sensitivity to speech intelligibility even with absent acoustic signals.
• A distributed cortical network, including auditory regions, modulates visual cortex processing of silent speech.
• Findings support a visuo-phonological mapping process occurring early in cortical processing.