一个多模式的基础模型,以帮助盲人和视力低下的人在环境互动中发挥作用
Yu Hao1, Fan Yang1, Hao Huang1
1Tandon School of Engineering, New York University, Brooklyn, NY 11201, USA.
Journal of imaging
|May 24, 2024
概括
这项研究引入了一种新的AI系统,以帮助失明和视力低下的人 (pBLV) 在不熟悉的环境中导航. 该系统使用基础模型来描述场景,识别物体和检测危险,提高独立性和安全性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 辅助技术 辅助技术 辅助技术
背景情况:
- 失明和视力低下 (pBLV) 的人在不熟悉的环境中面临着在场景识别和危险检测方面的重大挑战.
- 现有的辅助技术往往缺乏稳定性,需要广泛的培训,这限制了它们在动态现实世界的场景中的有效性.
研究的目的:
- 开发一个人工智能系统,帮助pBLV识别场景,识别物体,并在不熟悉的环境中检测掉的危险.
- 通过利用强大的基础模型来克服当前辅助技术的局限性.
主要方法:
- 使用大图像标记模型 (识别任何模型 - RAM) 进行初始对象识别.
- 采用快速工程来为pBLV需求量身定制系统响应,将对象识别与用户查询集成在一起.
- 利用视觉语言基础模型 (InstructBLIP) 来生成详细的环境描述和识别潜在风险.
主要成果:
- 拟议的方法可以准确地识别拍摄的图像中的物体.
- 该系统提供了对pBLV相关的环境的洞察性描述和分析.
- 室内和室外数据集的实验结果证明了该系统的有效性.
结论:
- 基础模型,结合快速工程,提供了一个强大的解决方案,以提高pBLV的视觉感知.
- 这种方法显著提高了pBLV导航和理解陌生的环境的能力.
- 开发的系统有望提高pBLV的独立性和安全性.
更多相关视频
09:29A Standardized Obstacle Course for Assessment of Visual Function in Ultra Low Vision and Artificial Vision
Published on: February 11, 2014
13.1K
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
323
相关概念视频
Sensory Modalities
1.3K
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
1.3K
Visual Agnosia
190
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
190
Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K
Gestalt Principles of Perception
292
Gestalt principles provide a framework for understanding how humans perceive objects as unified wholes within their context. These principles are essential in explaining the cognitive processes that make sense of complex visual stimuli by organizing them into coherent groups. One fundamental principle is proximity, which posits that objects located close to each other are perceived as a collective group. For instance, when dots are positioned near one another, the visual system interprets them...
292
Depth Perception and Spatial Vision
631
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
631
