Dynamic Crosswalk Scene Understanding for the Visually Impaired

Area of Science:

• Computer Vision
• Assistive Technology
• Robotics

Background:

• Visually impaired individuals face significant challenges with independent mobility, particularly in complex urban environments like crosswalks.
• Existing assistive technologies often lack the real-time environmental understanding needed for safe navigation.

Purpose of the Study:

• To develop and validate a novel system for dynamic crosswalk scene understanding to aid visually impaired individuals.
• To provide real-time audio feedback on crosswalk conditions, including object detection and traffic light status.

Main Methods:

• A head-mounted mobile system (SensingAI G1) with an Intel RealSense camera and cellphone was developed.
• The system detects key objects (crosswalk, vehicle, pedestrian) and pedestrian traffic light status.
• A comprehensive crosswalk scene understanding dataset was created for system validation.

Main Results:

• The proposed system demonstrated robust performance in understanding dynamic crosswalk scenes.
• Extensive experiments showed the system outperformed state-of-the-art approaches.
• The system successfully provided audio cues for safe road crossing indication.

Conclusions:

• The developed system offers a practical and useful solution for enhancing the independent mobility of visually impaired individuals.
• The system's ability to interpret complex crosswalk environments and provide actionable audio feedback is a significant advancement.