Related Experiment Videos
Hearing by eye: how much spatial degradation can be tolerated?
J MacDonald1, S Andersen, T Bachmann
1Department of Psychology, University of Portsmouth, Portsmough, PO1 2DY, UK. john.macdonald@port.ac.uk
Perception
|February 28, 2001
Summary
The McGurk effect, an auditory illusion, weakens with visual speech degradation. Even with coarse visual detail, auditory-visual interaction persists, indicating reliance on broad spatial information.
Area of Science:
- Auditory-visual perception
- Speech processing
- Psychophysics
Background:
- The McGurk effect demonstrates illusory auditory perception when visual lip movements conflict with auditory speech signals.
- Understanding the visual information's role in speech perception is crucial for audiology and human-computer interaction.
Purpose of the Study:
- To investigate the McGurk effect's tolerance to spatial degradation of visual speech information.
- To determine the spatial scale of visual information critical for auditory-visual speech perception.
Main Methods:
- Participants viewed videotaped faces with spatially degraded lip movements using a mosaic transform.
- The degree of spatial degradation was systematically varied.
- Participants reported their auditory perception and rated the clarity of the auditory stimulus.
Main Results:
- The McGurk illusion's strength systematically decreased as spatial quantisation coarseness increased.
- The illusion persisted even at the coarsest level of spatial degradation (11.2 pixels/face).
- Participants not experiencing the illusion still showed auditory-visual interaction effects in clarity ratings.
Conclusions:
- Auditory-visual interaction in visible speech perception relies on relatively coarse-spatial-scale visual information.
- The McGurk effect is robust to significant spatial degradation, highlighting the brain's efficient integration of multisensory speech cues.