Spatial and temporal factors during processing of audiovisual speech: a PET study

Area of Science:

• Neuroscience
• Cognitive Science
• Auditory Perception

Background:

• Speech perception integrates auditory and visual cues.
• Previous research focused on temporal aspects of audiovisual speech integration.
• Spatial factors in audiovisual speech processing require further investigation.

Purpose of the Study:

• To examine how temporal and spatial factors jointly influence brain activation during audiovisual speech processing.
• To identify distinct neural substrates for audiovisual synchrony and spatial integration.
• To understand the neural basis of the ventriloquism effect.

Main Methods:

• Positron Emission Tomography (PET) was used to image brain activity.
• Audiovisual speech stimuli were manipulated orthogonally for synchrony (synchronous vs. asynchronous) and location (same vs. different external locations).
• Brain activations were analyzed in relation to these temporal and spatial manipulations.

Main Results:

• Synchronous audiovisual speech increased activity in multisensory association areas (e.g., superior temporal sulcus) and some visual areas.
• Ventral occipital areas and superior temporal sulcus were sensitive to synchrony but not location.
• Lateral and dorsal occipital areas responded to synchronous stimuli at the same location.
• The right inferior parietal lobule was activated for synchronous stimuli at different locations, linked to the ventriloquism effect.

Conclusions:

• Different brain regions are involved in distinct aspects of audiovisual speech integration.
• Ventral brain areas are associated with audiovisual synchrony, influencing speech identification.
• Dorsal brain areas are linked to spatial multisensory interactions, including the ventriloquism effect.