对行走环境的自我中心感知 使用交互式视觉语言系统
概括
这项研究介绍了一种多式视觉语言系统,用于自我中心的感知,增强机器人技术的场景理解. 该系统以音频反生成个性化的图像字幕,改善了真实世界导航中的人类-AI交互.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
背景情况:
- 大型语言模型 (LLM) 提供了超越计算机视觉的上下文场景理解.
- 嵌入式智能和机器人从增强的感知系统中受益.
研究的目的:
- 开发一种多式视觉语言系统,用于自我中心的视觉感知.
- 启用个性化的图像标题和音频反,以实现现实世界的导航.
主要方法:
- 经过训练的基于变压器的视觉语言模型使用因果语言建模.
- 利用一个由43055个图像-文本对组成的自定义数据集,用于短暂的图像标题.
- 开发了一个语音合成模型和用户界面,通过用户提示提供音频反和个性化字幕.
主要成果:
- 生成了详细的图像标题 (例如: 10个单词) 具有高的ROUGE-L得分 (43.9%) 和低的单词错误率 (28.1%).
- 实现端到端处理时间为2.2秒.
- 通过用户提示显示标题的有效个性化.
结论:
- 多式联运系统提供准确,详细的场景描述.
- 个性化的标题优化了人类-人工智能交互的环境理解和导航.
- 这项工作通过将人类认知整合到生成模型中来推进体现的AI.
相关概念视频
Depth Perception and Spatial Vision
952
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
952
Perception
589
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
589
Visual System
700
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
700
Gestalt Principles of Perception
465
Gestalt principles provide a framework for understanding how humans perceive objects as unified wholes within their context. These principles are essential in explaining the cognitive processes that make sense of complex visual stimuli by organizing them into coherent groups. One fundamental principle is proximity, which posits that objects located close to each other are perceived as a collective group. For instance, when dots are positioned near one another, the visual system interprets them...
465
Vision
55.4K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.4K


