Related Experiment Video
Updated: Jun 10, 2026
09:04
Uncovering Beat Deafness: Detecting Rhythm Disorders with Synchronized Finger Tapping and Perceptual Timing Tasks
Published on: March 16, 2015
Audiovisual asynchrony detection in human speech
Joost X Maier1, Massimiliano Di Luca, Uta Noppeney
1Department of Psychology, Brandeis University, 415 South Street, MS 062, Waltham, MA 02453, USA. joost.maier@gmail.com
Summary
Humans show enhanced audiovisual speech perception, with a narrower temporal integration window for speech than for non-speech. This suggests speech perception is fine-tuned to natural audiovisual speech timing, tolerating visual but not auditory leading asynchronies.
Area of Science:
- Cognitive Science
- Neuroscience
- Speech Perception
Background:
- Audiovisual speech perception integrates visual and auditory information for enhanced intelligibility.
- This integration remains robust despite temporal offsets between visual and auditory speech signals.
Purpose of the Study:
- To characterize the temporal integration window for speech and non-speech stimuli.
- To investigate human adaptation to the specific temporal characteristics of natural audiovisual speech.
Main Methods:
- Manipulation of spectrotemporal structure, stimulus length, and task context.
- Comparison of temporal integration windows for speech versus non-speech stimuli with similar spectrotemporal properties.
Main Results:
- The temporal integration window for speech is narrower and more asymmetric than for non-speech signals.
- Audiovisual speech perception tolerates visual leading asynchronies but is highly sensitive to auditory leading asynchronies.
Conclusions:
- Human speech perception is fine-tuned to the natural statistics of audiovisual speech.
- This fine-tuning reflects the typical temporal order where facial movements precede acoustic speech articulation.
Related Concept Videos
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:21Perceiving Loudness, Pitch, and Location
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
01:15Auditory Pathway
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
01:31Hearing
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
01:01Perception of Sound Waves
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...