Related Experiment Video
Updated: Mar 23, 2026
16:31
Echo Particle Image Velocimetry
Published on: December 27, 2012
15.2K
Navigating from a Depth Image Converted into Sound
Chloé Stoll1, Richard Palluel-Germain1, Vincent Fristot2
1University Grenoble Alpes, LPNC, 38000 Grenoble, France; CNRS, LPNC, 38000 Grenoble, France.
Applied Bionics and Biomechanics
|March 29, 2016
Summary
This study shows that a sensory substitution device (SSD) using depth sensors can help blind individuals navigate. Performance improved over time, demonstrating the potential of this technology for the visually impaired.
Area of Science:
- Human-computer interaction
- Sensory substitution devices
- Assistive technology
Background:
- Depth sensors capture spatial data, mimicking human vision and touch.
- Sensory substitution devices (SSDs) convert this data into alternative sensory information, like sound.
- Previous research explored SSDs for various applications, including navigation.
Purpose of the Study:
- To evaluate a novel sound-based sensory substitution device (SSD) as a travel aid for visually impaired individuals.
- To assess the effectiveness of the MeloSee device in a real-world navigation task.
- To investigate the impact of a dual-task cognitive load on navigation performance.
Main Methods:
- A portable device, MeloSee, converted 2D depth images into real-time melodies.
- Sound intensity represented distance, stereo modulation indicated lateral position, and pitch conveyed verticality.
- Twenty-one blindfolded participants navigated four paths across two sessions, with some performing a concurrent tactile pattern recognition task.
Main Results:
- Participants successfully learned to use the MeloSee system for navigation on both familiar and new paths.
- Navigation performance, measured by travel time and errors, significantly improved between the two weekly sessions.
- The dual-task condition was manageable, with only a minor impact on navigation efficiency.
Conclusions:
- Kinect-type depth sensors show promise for developing effective SSDs for the visually impaired.
- Current limitations include restricted indoor usability and reduced efficiency at very short ranges.
- Further development is needed to optimize SSDs for diverse environments and distances.
Related Concept Videos
01:17Sound as Pressure Waves
4.7K
Sound waves, which are longitudinal waves, can be modeled as the displacement amplitude varying as a function of the spatial and temporal coordinates. As a column of the medium is displaced, its successive columns are also displaced. As the successive displacements differ relatively, a pressure difference with the surrounding pressure is created. The gauge pressure varies across the medium.
The pressure fluctuation depends on the difference in displacements between the successive points in the...
The pressure fluctuation depends on the difference in displacements between the successive points in the...
4.7K
01:01Perception of Sound Waves
6.0K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
6.0K
01:31Hearing
58.6K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
58.6K
01:05Intensity and Pressure of Sound Waves
1.9K
The intensity of sound waves can be related to displacement and pressure amplitudes by using their wave expressions and the definition of intensity. The critical step to achieve this is to write the power delivered by the particles on the wave as the product of force and velocity and simplify the force per unit area as the pressure. The velocity of the medium's particles can be derived from the displacement.
Unlike the time average of a sinusoidal term, which is zero since it is positive...
Unlike the time average of a sinusoidal term, which is zero since it is positive...
1.9K
01:12Korotkoff Sounds
12.4K
Korotkoff sounds are the specific sounds heard while measuring blood pressure using a sphygmomanometer, typically with a stethoscope or a Doppler device. They are named after Russian physician Nikolai Korotkov, who first described them in 1905. These sounds correspond to turbulent blood flow in the artery as the blood pressure cuff is gradually released after inflation.
During blood pressure assessment, inflating the cuff 30 millimeters of mercury above the patient's systolic blood pressure...
During blood pressure assessment, inflating the cuff 30 millimeters of mercury above the patient's systolic blood pressure...
12.4K
01:01Sound Waves
13.6K
Sound waves can be thought of as fluctuations in the pressure of a medium through which they propagate. Since the pressure also makes the medium's particles vibrate along its direction of motion, the waves can be modeled as the displacement of the medium's particles from their mean position.
Sound waves are longitudinal in most fluids because fluids cannot sustain any lateral pressure. In solids, however, shear forces help in propagating the disturbance in the lateral direction as well....
Sound waves are longitudinal in most fluids because fluids cannot sustain any lateral pressure. In solids, however, shear forces help in propagating the disturbance in the lateral direction as well....
13.6K