Munir: A Multimodal Smart-Glasses System for Enhancing Human-Computer Interaction for Visually Impaired Individuals.

Area of Science:

• Computer Science
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Visual impairment affects over 2.2 billion globally, with limited access to affordable assistive technologies.
• Existing solutions are often fragmented, expensive, and lack integrated multimodal capabilities.
• There is a need for inclusive, cost-effective assistive systems for daily living activities.

Purpose of the Study:

• To introduce Munir, an integrated multimodal assistive system for visually impaired users.
• To enhance human-computer interaction through a combination of mobile and smart glass technology.
• To provide real-time, inclusive support for daily living activities via a unified interface.

Main Methods:

• Developed an integrated system combining a mobile application and Bluetooth-enabled smart glasses.
• Utilized a hybrid machine learning architecture for real-time processing.
• Incorporated ten core capabilities including face recognition, OCR, and scene description.
• Implemented a bilingual (Arabic/English) voice interface and on-device processing for privacy.

Main Results:

• Achieved a 96.69% recognition rate with 0% False Accept Rate on the SCface dataset.
• Demonstrated a 4-5× cost advantage over commercial alternatives, with a first-year cost of $806.
• Ensured high responsiveness and user privacy through on-device processing.

Conclusions:

• Munir offers a scalable and affordable multimodal solution for visually impaired individuals.
• The system significantly enhances digital inclusion by providing accessible assistive technology.
• Munir represents a significant advancement in assistive technology for daily living support.