Related Experiment Video
Updated: Jul 15, 2026
13:51
Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion
Jyoti Mishra1, Antigona Martinez, Terrence J Sejnowski
1Division of Biological , University of California, San Diego, La Jolla, California 92093, USA.
Summary
This study reveals that perceiving an illusory second flash involves rapid cross-modal interactions between auditory and visual brain regions. Individual differences in neural connectivity influence the strength of this multisensory illusion.
Area of Science:
- Neuroscience
- Cognitive Science
- Psychophysics
Background:
- The illusory flash effect, where a single light flash appears doubled due to intervening auditory stimuli, is a well-documented phenomenon.
- Understanding the neural mechanisms and timing of this cross-modal illusion is crucial for insights into sensory integration.
Purpose of the Study:
- To investigate the temporal dynamics and cortical localization of neural processes underlying the illusory flash effect.
- To identify individual differences in neural connectivity related to cross-modal integration and illusion perception.
Main Methods:
- Utilized 64-channel electroencephalography (EEG) to record event-related potentials (ERPs) from 34 subjects.
- Analyzed difference ERPs to isolate neural activity specific to the illusory flash.
- Performed trial-by-trial analysis correlating neural activity with illusion perception.
Main Results:
- An early modulation of visual cortex activity (30-60 ms post-second sound) was observed, correlating with illusion frequency.
- Subjects experiencing the illusion more frequently showed consistent early visual modulation across stimuli, indicating stable individual differences in cross-modal integration.
- Neural activity patterns for the illusory flash differed from a real second flash, involving rapid interplay between auditory and visual cortices, including gamma bursts in the visual cortex.
Conclusions:
- Perception of the illusory flash relies on a rapid, dynamic interaction between auditory and visual cortical areas, triggered by the second auditory stimulus.
- Individual differences in neural connectivity significantly influence the susceptibility to this cross-modal illusion.
- The findings highlight the brain's sophisticated mechanisms for integrating information across different sensory modalities.
Related Concept Videos
01:27Synesthesia
Synesthesia is a remarkable condition where stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. People with synesthesia experience a blending or crossing of their senses, such as sight and sound, leading to cross-modal sensations. In this condition, the stimulation of one sense, such as hearing a number or musical note, triggers an experience of another sense, like sensing a specific color, taste, or smell. People...
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:14Motor and Sensory Areas of the Cortex
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.
01:01Perception of Sound Waves
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
01:15Auditory Pathway
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
01:12Visual Agnosia
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round end"...