Edge-Based Vision-Language Assistive System for the Visually Impaired: A Quantized VLM Approach

Area of Science:

• Artificial Intelligence
• Assistive Technology
• Computer Vision

Background:

• Cloud-based image captioning systems for the visually impaired face challenges like latency and limited contextual information.
• Existing solutions often lack the real-time processing and on-device capabilities required for effective real-world navigation.

Purpose of the Study:

• To develop a novel, edge-deployable assistive system for visually impaired individuals.
• To overcome the limitations of cloud-based solutions by enabling on-device, real-time scene interpretation.
• To enhance user independence and interaction through accessible AI.

Main Methods:

• Integration of a quantized LLaVA-NeXT-13B vision-language model (VLM) with speech recognition (Whisper) and synthesis (PIPER).
• Deployment on a wearable platform (NVIDIA Jetson Orin NX and Raspberry Pi) for self-contained operation.
• User-centered design with a button interface and Bluetooth audio output to minimize cognitive load.

Main Results:

• Quantization of the VLM showed minimal accuracy loss on VizWiz-VQA (2.6%) and VQAv2 (0.9%).
• User evaluations revealed a 25% increase in image identification accuracy compared to a baseline model.
• The system achieved practical latency (4-5 seconds) and high usability scores.

Conclusions:

• The developed edge-deployable VLM system offers a viable, real-time solution for visually impaired individuals.
• On-device processing significantly improves accessibility and independence by removing cloud reliance.
• This work contributes to scalable, interpretable, and accessible AI-driven assistive technologies.