Related Experiment Video
Updated: Nov 19, 2025
13:51
Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
20.2K
Modulation of early auditory processing by visual information: Prediction or bimodal integration?
Maria V Stuckenberg1,2, Erich Schröger3, Andreas Widmann3,4
1Institute of Psychology, Leipzig University, Neumarkt 9-19, D-04109, Leipzig, Germany. maria_viktoria.stuckenberg@uni-leipzig.de.
Attention, Perception & Psychophysics
|January 28, 2021
Summary
Visual cues can mislead auditory expectations, triggering an incongruency response (IR) in brain activity. This study found the IR, a brain potential reflecting prediction errors, only occurs when visual and auditory stimuli are presented asynchronously.
Area of Science:
- Neuroscience
- Auditory Perception
- Cognitive Psychology
Background:
- An early visuo-auditory incongruency response (IR) in auditory event-related potentials (ERPs) occurs 100 ms after sound onset when a visual cue mismatches auditory expectations.
- This IR has been attributed to a mismatch between predicted and actual auditory input, suggesting it requires asynchronous stimulus presentation.
- An alternative hypothesis proposes that integrated bimodal representations could lead to an IR even with synchronous presentation due to violated visual-auditory relationships.
Purpose of the Study:
- To investigate whether the incongruency response (IR) to visuo-auditory mismatches is specific to asynchronous stimulus presentation or also occurs with synchronous presentation.
- To test the sensorial prediction error hypothesis regarding the timing and conditions of the IR.
Main Methods:
- Participants were presented with visual note symbols (above/below a cross) and auditory tones (high/low pitch) in both asynchronous and synchronous conditions.
- Stimuli were designed with a high probability of congruence (90%) and a low probability of incongruence (10%) to establish strong associations.
- Auditory event-related potentials (ERPs) were recorded and analyzed to identify incongruency effects, specifically the IR, N2, and P3 components.
Main Results:
- In both asynchronous and synchronous conditions, incongruent tones elicited N2 and P3 ERP components, indicating a general mismatch detection.
- However, the early auditory incongruency response (IR), occurring around 100 ms post-stimulus, was exclusively observed in the asynchronous presentation condition.
- The findings support the sensorial prediction error hypothesis, suggesting the IR reflects a rapid update of auditory sensory predictions based on preceding visual cues.
Conclusions:
- The incongruency response (IR) is a sensitive indicator of prediction errors in auditory processing, specifically when visual cues precede auditory stimuli.
- The absence of the IR in the synchronous condition suggests that the early auditory ERP modulation relies on temporal prediction rather than integrated bimodal representations.
- This research clarifies the temporal dynamics of visuo-auditory integration and prediction error signaling in the brain.
Related Concept Videos
01:20Parallel Processing
441
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
441
01:31Hearing
55.4K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
55.4K
01:17Auditory Perception
788
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
788
01:15Auditory Pathway
6.5K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
6.5K
01:13The Cochlea
48.9K
The cochlea is a coiled structure in the inner ear that contains hair cells—the sensory receptors of the auditory system. Sound waves are transmitted to the cochlea by small bones attached to the eardrum called the ossicles, which vibrate the oval window that leads to the inner ear. This causes fluid in the chambers of the cochlea to move, vibrating the basilar membrane.
48.9K
01:26Visual System
1.4K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
1.4K