A Mobile Application for Keyword Search in Real-World Scenes

Area of Science:

• Human-Computer Interaction
• Assistive Technology
• Computer Vision

Background:

• Keyword search in visually cluttered environments poses challenges, particularly for individuals with low vision.
• Conventional magnification aids for low vision constrict the field of view, hindering efficient visual search.
• Many visual search tasks involve knowing the target keyword but not its location within a scene.

Purpose of the Study:

• To develop and evaluate a mobile application that assists users in locating keywords within complex visual scenes.
• To compare the effectiveness of the developed mobile application against traditional optical magnifiers for visual search tasks.

Main Methods:

• Development of a mobile application incorporating voice/text input, optical character recognition (OCR), and image zooming capabilities.
• Evaluation of various mainstream OCR engines for application performance.
• A user study comparing the mobile application with a handheld magnifier, using normally sighted adults with induced visual acuity reduction.

Main Results:

• The mobile application demonstrated significantly faster search times than handheld magnifiers for difficult visual search tasks.
• No significant difference in search times was observed between the app and magnifier for easier tasks.
• The study evaluated the performance of different OCR engines integrated into the application.

Conclusions:

• The developed mobile application offers a more efficient solution for keyword search in cluttered environments for individuals with low vision compared to optical magnifiers, especially for challenging tasks.
• The application leverages OCR technology to overcome the limitations of field-of-view constriction inherent in magnification aids.
• Future research could further refine OCR integration and explore diverse user populations and search scenarios.