Visual Echolocation Concept for the Colorophone Sensory Substitution Device Using Virtual Reality.

Area of Science:

• Human-computer interaction
• Assistive technology
• Sensory substitution

Background:

• Visually impaired individuals face significant challenges in perceiving 3D environments, crucial for navigation.
• Existing Electronic Travel Aids (ETAs) often lack comprehensive 2D, 3D, and color information integration.
• Current devices typically offer limited field-of-view data or focus on specific image points.

Purpose of the Study:

• To propose a novel visual echolocation concept with integrated color sonification.
• To extend the capabilities of the Colorophone assistive device for the visually impaired.
• To provide a system that sonifies 2D, 3D, and color information for the entire scene.

Main Methods:

• Simulation of a visual echolocation concept using VIRCO (Virtual Reality training and evaluation system).
• Development of an algorithm for sonifying color and distance data from a virtual 3D scene.
• Conducting an introductory user evaluation in a virtual reality environment.

Main Results:

• Demonstrated the feasibility of sonifying color and distance for an entire scene.
• Proof-of-concept simulation successfully validated the visual echolocation approach.
• User evaluation indicated effectiveness in perceiving object distance, position, and color in VR.

Conclusions:

• The proposed visual echolocation concept with color sonification is a viable approach for enhancing environmental perception for the visually impaired.
• The developed algorithm shows potential for implementation on hardware-based stereo camera platforms.
• Further development and user testing are warranted to refine the system for real-world application.