CA-VLN:在MLLM驱动的视觉语言导航中的协作代理
Ruolin Zhu1, Shaobin Li1, Zixing Zhu1
1School of Information and Communication Engineering, Communication University of China, Beijing 100024, China.
Sensors (Basel, Switzerland)
|February 27, 2026
概括
本研究介绍了视觉语言导航 (CA-VLN) 中的协作代理,这是一种使用多式联络大型语言模型改进机器人在未见环境中的导航的新框架. CA-VLN通过语义理解和基于记忆的规划的协作代理来提高概括性和成功率.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 人工智能的人工智能
- 计算机视觉 计算机视觉
背景情况:
- 将机器人导航推广到新的,看不见的环境是视觉语言导航 (VLN) 中的一个重大挑战.
- 现有的方法经常与长远的规划和整合常识推理进行斗争,以实现稳健的导航.
研究的目的:
- 开发一个新的框架,以提高视觉语言导航中的概括能力.
- 为了提高导航性能,利用多式大型语言模型的世界知识.
- 为了使机器人能够在以前未被观察到的环境中成功导航.
主要方法:
- 在视觉语言导航 (CA-VLN) 中引入协作代理,一个双代理架构.
- 知识代理将语义上下文和常识推理集成到行动预测中.
- 一个层次历史代理建立了详细的情节记忆,用于长期规划.
主要成果:
- 在已建立的基准 (R2R,REVERIE,SOON) 上,CA-VLN 实现了最先进的性能.
- 观察到对未见的环境进行概括的显著改进.
- 在新奇而复杂的环境中提高了导航成功率.
结论:
- 拟议的CA-VLN框架有效地利用多式联网大型语言模型进行高级导航.
- 双代理架构使语义理解和情节性记忆之间的动态相互作用成为可能.
- 在解决视觉语言导航中的泛化挑战方面,CA-VLN代表了重大进展.
更多相关视频
07:09Integrating Visual Psychophysical Assays within a Y-Maze to Isolate the Role that Visual Features Play in Navigational Decisions
Published on: May 2, 2019
6.5K
06:28A Networked Desktop Virtual Reality Setup for Decision Science and Navigation Experiments with Multiple Participants
Published on: August 26, 2018
6.3K
相关概念视频
Visual Agnosia
1.4K
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
1.4K
Visual System
2.1K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
2.1K
Components of Language
866
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
866
Purposive Learning
545
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
545
Indirect Motor Pathways
3.7K
The indirect motor or extrapyramidal pathways originate in the brainstem, the lower portion of the brain that connects it to the spinal cord. They consist of several distinct tracts, each with specialized functions. The four main tracts of the indirect motor pathways are the vestibulospinal tract, the reticulospinal tract, the tectospinal tract, and the rubrospinal tract.
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
The vestibulospinal tract originates in the vestibular nuclei of the brainstem. The vestibular system detects changes in...
3.7K
Fluid Mosaic Model
18.5K
Scientists identified the plasma membrane in the 1890s and its principal chemical components (lipids and proteins) by 1915. The model for plasma membrane structure, proposed in 1935 by Hugh Davson and James Danielli, was the first model to be widely accepted in the scientific community. The model was based on the plasma membrane's "railroad track" appearance in early electron micrographs. Davson and Danielli theorized that the plasma membrane's structure resembled a sandwich...
18.5K
