Related Experiment Video
Updated: May 23, 2025
09:13
Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder
Published on: April 22, 2015
16.4K
Primitive audiovisual integration of speech.
Robert E Remez1, Kathryn R Dubowski2, Daria F Ferro3
1Department of Psychology and Program in Cognitive Science, Barnard College, Columbia University, 3009 Broadway, New York, NY, 10027-6598, USA. remez@columbia.edu.
Attention, Perception & Psychophysics
|March 7, 2025
Summary
Audiovisual sensory integration makes unintelligible speech intelligible. This process, crucial for speech perception, relies on early, non-phonetic integration between sight and sound, even with imperfect signals.
Area of Science:
- Cognitive Science
- Auditory Perception
- Visual Perception
Background:
- Speech perception typically relies on both auditory and visual cues.
- Understanding how the brain integrates information from different senses is key to cognitive science.
Purpose of the Study:
- To investigate the mechanisms of audiovisual sensory integration in speech perception.
- To determine the role of phonetic and non-phonetic information in this integration process.
Main Methods:
- Presenting unintelligible visual speech with a synchronized, unintelligible acoustic sinusoid.
- Testing intelligibility under conditions of varying synchrony and sensory stream lead-lag.
- Analyzing the contribution of unimodal and bimodal Gestalt principles.
Main Results:
- Audiovisual integration rendered the speech intelligible, demonstrating robust sensory fusion.
- Integration occurred even without resolving linguistic details in individual modalities.
- Integration persisted despite desynchronization, particularly when visual input preceded auditory input.
Conclusions:
- Early, non-phonetic sensory integration plays a significant role in audiovisual speech perception.
- The temporal dynamics of sensory input critically influence audiovisual speech intelligibility.
- Audiovisual coherence can be achieved even with unnatural auditory components, highlighting the brain's integrative power.
Related Concept Videos
01:26Visual System
468
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
468
01:10Higher Mental Functions of the Brain: Language
706
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
706
01:15Auditory Pathway
4.6K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
4.6K
01:31Hearing
51.7K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
51.7K
01:14Motor and Sensory Areas of the Cortex
2.7K
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor...
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor...
2.7K
01:24Vision
52.9K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.9K