从视觉中看不见:使用基础模型重写观察指令,以增强视觉语言导航
IEEE transactions on neural networks and learning systems
|November 10, 2025
概括
视觉语言导航 (VLN) 中的数据稀缺性是由重写驱动的增强 (RAM) 解决的. 通过重写现有示例,RAM生成新的训练数据,改善对未见的环境的代理通用化,无需模拟器或大量手工劳动.
科学领域:
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
- 计算机视觉 计算机视觉
背景情况:
- 数据稀缺是视觉语言导航 (VLN) 的一个主要限制,阻碍了对新环境的代理通用化.
- 现有的方法使用模拟器或网络数据,其多样性有限或需要大量的手动清洁.
- 这限制了特工在现实世界中有效导航,看不见的场景的能力.
研究的目的:
- 为 VLN 引入一种新的重写驱动增强 (RAM) 范式.
- 通过从现有数据中生成多样化,未见的观察指令对来克服数据稀缺性.
- 以无模拟器和节省劳动力的方式提高VLN代理的概括能力.
主要方法:
- 用视觉语言模型 (VLM) 和大语言模型 (LLM) 来创建多样化的场景描述.
- 文本到图像生成模型 (T2IMs) 基于重写的描述来合成新的观察结果.
- 观察对比指令重写使用LLM将新指令与合成的观察对齐.
- 混合然后集中训练策略与随机观察作物增强数据多样性并减少噪音.
主要成果:
- RAM 范式成功地为 VLN 训练生成了新的观察指令对.
- 实验表明,在离散 (R2R,REVERIE,R4R) 和连续 (R2R-CE) VLN 数据集上,性能优越,概括性强.
- 该方法有效地提高了在未见的环境中代理的性能.
结论:
- 重写驱动的增强 (RAM) 是一种有效的方法来解决VLN中的数据稀缺问题.
- 拟议的方法为数据增强提供了无模拟器和节省劳动力的解决方案.
- RAM显著提高了VLN代理的泛化能力,为更强大的导航系统铺平了道路.
相关概念视频
Observational Learning
804
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
804
Vision
59.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.3K
Depth Perception and Spatial Vision
1.8K
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
1.8K
Visual Agnosia
920
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
920
Purposive Learning
428
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
428


