A wearable obstacle avoidance device for visually impaired individuals with cross-modal learning

Area of Science:

• Wearable technology
• Robotics
• Computer vision

Background:

• Wearable obstacle avoidance devices face challenges in balancing reliability, response time, battery life, and usability.
• Existing systems often struggle to meet all practical demands simultaneously.

Purpose of the Study:

• To develop a wearable obstacle avoidance device that integrates high reliability, rapid response, extended duration, and a usable design.
• To enhance environmental perception and obstacle detection for improved user safety.

Main Methods:

• A self-developed wearable device comprising smart glasses (400g, 80g battery) and a smartphone.
• Multi-modal data collection (video, depth) with a depth-aided video compression module on a custom FPGA board.
• Cross-modal obstacle detection module on a smartphone utilizing cross-modal learning for alerts.

Main Results:

• Demonstrated 100% collision avoidance rates in various indoor and outdoor experiments.
• Achieved a system delay of less than 320 milliseconds.
• Provided approximately 11 hours of operational duration on a single charge.

Conclusions:

• The developed wearable obstacle avoidance device effectively addresses the limitations of current systems.
• The system offers a reliable, responsive, and long-lasting solution for obstacle avoidance, enhancing user safety through multi-modal perception and cross-modal learning.