Related Experiment Video
Updated: Sep 13, 2025

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind
Published on: March 27, 2013
Research and Implementation of Travel Aids for Blind and Visually Impaired People
Jun Xu1,2, Shilong Xu1,2, Mingyu Ma1,2
1School of Automation, Harbin University of Science and Technology, Harbin 150080, China.
Abstract:
Blind and visually impaired (BVI) people face significant challenges in perception, navigation, and safety during travel. Existing infrastructure (e.g., blind lanes) and traditional aids (e.g., walking sticks, basic audio feedback) provide limited flexibility and interactivity for complex environments. To solve this problem, we propose a real-time travel assistance system based on deep learning. The hardware comprises an NVIDIA Jetson Nano controller, an Intel D435i depth camera for environmental sensing, and SG90 servo motors for feedback. To address embedded device computational constraints, we developed a lightweight object detection and segmentation algorithm. Key innovations include a multi-scale attention feature extraction backbone, a dual-stream fusion module incorporating the Mamba architecture, and adaptive context-aware detection/segmentation heads. This design ensures high computational efficiency and real-time performance. The system workflow is as follows: (1) the D435i captures real-time environmental data; (2) the processor analyzes this data, converting obstacle distances and path deviations into electrical signals; (3) servo motors deliver vibratory feedback for guidance and alerts. Preliminary tests confirm that the system can effectively detect obstacles and correct path deviations in real time, suggesting its potential to assist BVI users. However, as this is a work in progress, comprehensive field trials with BVI participants are required to fully validate its efficacy.
More Related Videos
09:29A Standardized Obstacle Course for Assessment of Visual Function in Ultra Low Vision and Artificial Vision
Published on: February 11, 2014
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025