DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational...

Area of Science:

• Computer Vision
• Deep Learning
• Assistive Technology

Background:

• Real-time object detection and tracking are crucial for autonomous navigation and assistive devices.
• Existing methods often lack robustness in complex outdoor environments or require prior object knowledge.
• Visually impaired (VI) individuals face significant challenges navigating crowded urban scenes.

Purpose of the Study:

• To introduce a novel framework leveraging computer vision and deep convolutional neural networks (CNNs) for real-time object detection, tracking, and recognition.
• To develop an assistive device for VI individuals to improve their environmental awareness and safety during navigation.
• To validate the performance and robustness of the proposed framework and assistive device.

Main Methods:

• An object detection technique identifies static and dynamic objects without prior assumptions on their characteristics.
• A novel object tracking method utilizes two offline-trained CNNs, alternating between motion-based tracking and visual similarity prediction.
• The framework is integrated into an assistive device tested with VI users in real-world urban scenarios.

Main Results:

• The object tracking method achieves state-of-the-art results on standard VOT datasets with minimized computational complexity.
• The assistive device demonstrates high accuracy (>90%) and robustness (>90%) in diverse and dynamic urban scenes.
• Validation on a dataset of 30 videos acquired with VI users confirms the system's effectiveness.

Conclusions:

• The proposed framework offers an efficient and accurate solution for real-time object detection and tracking in outdoor environments.
• The integrated assistive device significantly enhances the navigation capabilities and safety of visually impaired individuals.
• The system's performance indicates its potential for practical application in aiding VI people.