Related Experiment Video
Updated: Jul 1, 2026
12:43
A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS
Published on: February 21, 2011
34.3K
Neural speech tracking in a virtual acoustic environment: audio-visual benefit for unscripted continuous speech
Mareike Daeglau1, Jürgen Otten2, Giso Grimm2
1Neuropsychology Lab, Department of Psychology, Carl von Ossietzky University of Oldenburg, Oldenburg, Germany.
Frontiers in Human Neuroscience
|April 24, 2025
Summary
The audio-visual benefit enhances speech perception, especially in noisy environments. Natural speech and lip movements are crucial for understanding, highlighting individual speaker differences in real-world listening.
Area of Science:
- Neuroscience
- Speech Perception
- Acoustic Auditory Science
Background:
- The audio-visual benefit, where visual cues enhance speech understanding, is well-established.
- Previous research often uses controlled lab settings with scripted speech.
Purpose of the Study:
- To investigate the audio-visual benefit using natural, unscripted speech in a virtual acoustic environment.
- To assess neural responses and isolate the role of lip movements in speech perception.
Main Methods:
- Utilized electroencephalography (EEG) for cortical speech tracking.
- Compared audio-visual, audio-only, visual-only, and masked-lip conditions.
- Analyzed acoustic and visual speaker features like pitch and lip-openness.
Main Results:
- Confirmed significant audio-visual enhancement in speech tracking, particularly with background noise.
- Demonstrated that masked-lip conditions yielded results similar to audio-only, stressing lip movement importance.
- Showcased the feasibility of cortical speech tracking with naturalistic stimuli.
Conclusions:
- Lip movements are vital for speech perception in challenging listening conditions.
- Individual speaker characteristics influence audio-visual integration in real-world scenarios.
- Cortical speech tracking is viable with naturalistic speech, offering insights into auditory-visual processing.
Keywords:
EEGcontinuous speechneural speech trackingunscripted conversationvirtual acoustic environmentMore Related Videos
Related Concept Videos
01:01Perception of Sound Waves
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:21Perceiving Loudness, Pitch, and Location
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...