Related Experiment Video
Updated: Jun 22, 2026

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
The integration processing of the visual and auditory information in videos of real-world events: an ERP study
Baolin Liu1, Zhongning Wang, Zhixing Jin
1Department of Computer Science and Technology, Tsinghua University, Beijing 100084, PR China. liubaolin@tsinghua.edu.cn
Abstract:
In real life, the human brain usually receives information through visual and auditory channels and processes the multisensory information, but studies on the integration processing of the dynamic visual and auditory information are relatively few. In this paper, we have designed an experiment, where through the presentation of common scenario, real-world videos, with matched and mismatched actions (images) and sounds as stimuli, we aimed to study the integration processing of synchronized visual and auditory information in videos of real-world events in the human brain, through the use event-related potentials (ERPs) methods. Experimental results showed that videos of mismatched actions (images) and sounds would elicit a larger P400 as compared to videos of matched actions (images) and sounds. We believe that the P400 waveform might be related to the cognitive integration processing of mismatched multisensory information in the human brain. The results also indicated that synchronized multisensory information would interfere with each other, which would influence the results of the cognitive integration processing.
More Related Videos
11:39Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
10:30Simultaneous Recording of Electroretinography and Visual Evoked Potentials in Anesthetized Rats
Published on: July 1, 2016
Related Concept Videos
Parallel Processing
Visual System
Once through the pupil, the light passes through the lens, a...
Integration of Synaptic Events
Vision
Motor and Sensory Areas of the Cortex
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.
Perception
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...