Related Experiment Video
Updated: Jun 8, 2026
09:13
Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder
Published on: April 22, 2015
Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources
Céline Cappe1, Gregor Thut, Vincenzo Romei
1Neuropsychology and Neurorehabilitation Service, Department of Clinical Neurosciences, Centre Hospitalier Universitaire Vaudois and University of Lausanne, 1011 Lausanne, Switzerland. celine.cappe@chuv.ch
Summary
Auditory-visual interactions in the human brain involve nonlinear, topographic modulations in early processing stages. These multisensory effects engage distinct neural configurations, offering insights into brain organization across species.
Area of Science:
- Neuroscience
- Cognitive Science
- Sensory Processing
Background:
- Current models propose multisensory interactions in early cortical processing, but human auditory-visual (AV) interactions remain debated.
- Conventional event-related potential (ERP) analysis faces challenges in determining precise timing, directionality, and neurophysiological interpretability of AV interactions.
- Hemodynamic imaging and TMS offer general support but lack detailed temporal and source localization for AV interactions.
Purpose of the Study:
- To investigate the neurophysiological underpinnings of human auditory-visual interactions using novel ERP analysis methods.
- To clarify the timing, nature (nonlinear/linear), and brain regions involved in early AV processing.
- To reconcile findings across different neuroimaging modalities and species.
Main Methods:
- Recorded human ERPs to attended, task-irrelevant auditory and visual stimuli.
- Applied novel ERP signal analysis techniques, including source estimations.
- Utilized statistical analyses to identify neural generators and their activity patterns.
Main Results:
- Nonlinear AV interactions were identified between 60-95 ms post-stimulus, driven by topographic modulations rather than amplitude changes.
- Distinct configurations of intracranial generators, not simple amplitude modulation, underlie AV stimulus processing.
- Primary visual, primary auditory, and posterior superior temporal regions were identified as key mediators, showing subadditive nonlinear effects.
Conclusions:
- Nonlinear, topographic modulations in early processing stages are crucial for human auditory-visual interactions.
- AV interactions involve distinct neural configurations in specific brain regions, including primary sensory cortices.
- These findings provide a noninvasive method to study multisensory interactions in humans, aligning with primate research and advancing cross-modal synthesis.
Related Concept Videos
01:20Parallel Processing
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
01:27Synesthesia
Synesthesia is a remarkable condition where stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. People with synesthesia experience a blending or crossing of their senses, such as sight and sound, leading to cross-modal sensations. In this condition, the stimulation of one sense, such as hearing a number or musical note, triggers an experience of another sense, like sensing a specific color, taste, or smell. People...
01:15Auditory Pathway
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:14Motor and Sensory Areas of the Cortex
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.