Related Experiment Video
Updated: Aug 10, 2025
11:39
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
2.2K
Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution.
Camille Bordeau1, Florian Scalvini2, Cyrille Migniot2
1LEAD-CNRS UMR5022, Université de Bourgogne, Dijon, France.
Frontiers in Psychology
|February 13, 2023
Summary
Pitch-based auditory cues improve elevation perception for blind individuals using visual-to-auditory sensory substitution devices. This method enhances spatial awareness more effectively than traditional sound spatialization techniques.
Area of Science:
- Auditory Perception
- Assistive Technology
- Human-Computer Interaction
Background:
- Visual-to-auditory sensory substitution devices aid the blind by translating visual information into soundscapes.
- Virtual Acoustic Space (VAS) with Head Related Transfer Functions (HRTFs) is used for spatial sound localization, but elevation perception accuracy is limited.
- Audiovisual cross-modal correspondence between pitch and visual elevation offers an alternative for conveying elevation information.
Purpose of the Study:
- To compare the effectiveness of different elevation encoding methods in visual-to-auditory sensory substitution devices.
- To evaluate early localization abilities using spatialization-based (Noise) versus pitch-based (Monotonic, Harmonic) auditory cues.
- To assess the impact of familiarization on object localization performance.
Main Methods:
- Thirty-eight blindfolded participants localized virtual targets using soundscapes.
- Localization tasks were performed before and after familiarization with different visual-to-auditory encoding methods.
- Methods compared included spatialization-based (Noise) and pitch-based (Monotonic, Harmonic) elevation encoding.
Main Results:
- Participants demonstrated significantly higher accuracy in localizing elevation with pitch-based encodings compared to the spatialization-based method.
- Azimuth localization performance showed only minor differences across the tested encoding methods.
- Familiarization with the devices improved overall localization accuracy.
Conclusions:
- Pitch-based encoding is more intuitive and effective for conveying elevation information in visual-to-auditory sensory substitution.
- Cross-modal correspondence between pitch and elevation facilitates spatial perception, especially with non-individualized spatialization.
- These findings suggest promising avenues for improving assistive technologies for the visually impaired.
Related Concept Videos
01:21Perceiving Loudness, Pitch, and Location
302
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
302
01:15Sensory Modalities
1.5K
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
1.5K
01:17Auditory Perception
408
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
408
01:15Auditory Pathway
5.6K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
5.6K
01:13The Cochlea
45.5K
The cochlea is a coiled structure in the inner ear that contains hair cells—the sensory receptors of the auditory system. Sound waves are transmitted to the cochlea by small bones attached to the eardrum called the ossicles, which vibrate the oval window that leads to the inner ear. This causes fluid in the chambers of the cochlea to move, vibrating the basilar membrane.
45.5K
01:31Hearing
52.8K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
52.8K