一项关于使用文本描述来视觉识别未知的场景的生成模型的研究
Jose Martinez-Carranza1, Delia Irazú Hernández-Farías1, Victoria Eugenia Vazquez-Meza1
1Department of Computational Science, Instituto Nacional de Astrofisica, Optica y Electronica (INAOE), Puebla 72840, Mexico.
Sensors (Basel, Switzerland)
|November 14, 2023
概括
生成型模型和多模式嵌入有助于人工智能 (如配送无人机) 从文本描述中可视化未知的位置. 这项技术增强了机器人在陌生的环境中导航的场景识别.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 机器人技术 机器人技术 机器人技术
背景情况:
- 人工代理需要强大的方法来导航和识别不熟悉的环境.
- 当前的方法往往依赖于先前存在的地图或直接的视觉输入,限制了它们在新情况下的实用性.
研究的目的:
- 研究使用生成模型和多模式嵌入表示的方法,使人工智能能够从文本描述中可视化不熟悉的目的地.
- 评估将图像生成,文本生成和文本增强策略用于场景识别的有效性.
主要方法:
- 利用像稳定扩散这样的生成模型从文本创建图像.
- 采用了嵌入式表示,如CLIP和VisualBERT,用于比较生成和现实世界的场景图像.
- 实施文本增强技术,包括ChatGPT,以完善文本描述以进行评估.
主要成果:
- 证明了生成模型能够从文本场景描述中产生相关的视觉表示的能力.
- 展示了生成工具和多模式嵌入之间的协同作用,以提高人工智能代理的场景识别精度.
- 验证了文本增强在为机器人导航创建简洁和信息化的描述中的有效性.
结论:
- 将生成模型与多模式嵌入相结合,可以显著提高人工智能对基于文本识别未知的场景的能力.
- 这种方法为自主系统提供了实际的解决方案,特别是在无人机包裹配送和服务机器人中,这些机器人在未绘制的地区运行.
- 未来的应用包括使机器人能够仅使用文字指导来导航和与环境互动.
更多相关视频
08:25Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.0K
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.7K
相关概念视频
01:12Visual Agnosia
210
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
210
01:24Prosopagnosia
181
Prosopagnosia, also known as face blindness, is the inability to recognize faces. In severe cases, individuals with prosopagnosia may not recognize close family members, including parents and spouses, by their faces. For instance, someone with prosopagnosia might walk past their child in a crowd, only realizing their mistake upon noticing their child's distinctive backpack or favorite jacket. Prosopagnosia specifically impairs facial recognition, while the recognition of other objects or...
181
01:26Visual System
591
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
591
01:24Vision
53.5K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.5K
02:16Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
01:22Photoreceptors and Visual Pathways
6.1K
At the molecular level, visual signals trigger transformations in photopigment molecules, resulting in changes in the photoreceptor cell's membrane potential. The photon's energy level is denoted by its wavelength, with each specific wavelength of visible light associated with a distinct color. The spectral range of visible light, classified as electromagnetic radiation, spans from 380 to 720 nm. Electromagnetic radiation wavelengths exceeding 720 nm fall under the infrared category,...
6.1K