Deep learning based object detection and surrounding environment description for visually impaired people

Area of Science:

• Computer Vision and Machine Learning
• Assistive Technologies for the Visually Impaired

Background:

• Object detection is crucial for identifying and locating objects in visual data.
• Existing assistive devices for the visually impaired have limitations in providing comprehensive environmental awareness.

Purpose of the Study:

• To develop a low-cost, deep learning-based assistive system for obstacle detection and environmental depiction for visually impaired people.
• To enhance the independence and safety of visually impaired individuals through real-time audio feedback.

Main Methods:

• Utilized TensorFlow object detection API with a pre-trained SSDLite MobileNetV2 model.
• Optimized the MobileNetV2 model using gradient particle swarm optimization (PSO).
• Integrated Raspberry Pi 4B, camera, and audio modules (Google text-to-speech, PyAudio, playsound, speech recognition) for real-time processing and feedback.

Main Results:

• Developed a real-time object detection system capable of identifying 90 different objects.
• Created an "ambiance mode" using transfer learning to describe the surrounding environment (weather conditions).
• Evaluated system performance on desktop and embedded Raspberry Pi systems, measuring accuracy, frame rate, and other key metrics.

Conclusions:

• The proposed low-cost system effectively aids visually impaired individuals in detecting obstacles and understanding their surroundings.
• This assistive technology offers a significant improvement over traditional methods like white canes.
• The system has the potential to greatly improve the daily lives and independence of visually impaired users.