Related Experiment Video
Updated: Apr 19, 2026
11:39
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
2.7K
Using an auditory sensory substitution device to augment vision: evidence from eye movements
Thomas D Wright1, Aaron Margolis, Jamie Ward
1School of Psychology and Sackler Centre for Consciousness Science, University of Sussex, Falmer, Brighton, BN1 9QH, UK.
Experimental Brain Research
|December 17, 2014
Summary
This study shows that combining vision with sound-from-vision (soundscapes) can improve locating hidden auditory targets. Visual background information enhances detection, even when the target is only heard.
Area of Science:
- Neuroscience
- Sensory Perception
- Human-Computer Interaction
Background:
- Sensory substitution devices translate sensory input (e.g., vision to sound) to aid individuals with sensory impairments.
- Multimodal sensory integration research explores combining different sensory inputs for enhanced perception.
Purpose of the Study:
- To investigate the benefits of simultaneously presenting natural vision and sound-from-vision (soundscapes) for target localization.
- To determine if visual background information can enhance the detection of an auditory target presented within a soundscape.
- To analyze eye-tracking data to understand how visual and auditory cues influence search behavior.
Main Methods:
- Participants performed a target localization task using simultaneous visual and auditory stimuli (soundscapes).
- A discrepancy was introduced where the auditory target was not visually present.
- Behavioral responses and eye movements (via eye-tracking) were recorded.
Main Results:
- Behavioral performance in detecting the auditory target improved when non-target visual background information was present.
- Eye-tracking revealed participants fixated longer on the target's quadrant, even if their final judgment differed.
- Visual information appears to facilitate auditory target detection by aiding predictive processing of the soundscape.
Conclusions:
- Multimodal sensory substitution, combining vision with soundscapes, can enhance target detection and localization.
- This approach shows potential for aiding visually impaired individuals with residual vision.
- Findings suggest applications in guiding visual search within complex environments for sighted individuals.
