整合神经辐射场端到端用于认知视觉运动导航
IEEE transactions on pattern analysis and machine intelligence
|September 6, 2024
概括
这项研究介绍了一种使用神经辐射场 (NeRF) 来增强空间认知的新型视觉运动导航系统. 该框架具有结构辐射注意力 (SRA),在现实世界机器人导航任务中实现了卓越的性能.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 计算机视觉 计算机视觉
- 人工智能的人工智能
背景情况:
- 现有的导航系统通常依赖于模块化设计和明确的场景表示.
- 整合像神经辐射场 (NeRF) 这样的隐性空间表示来实现体现的决策仍然是一个挑战.
研究的目的:
- 提出一个端到端的视觉导航框架,利用NeRF进行空间认知.
- 引入一种用于从NeRF内存中提取关键决策信息的新机制,用于嵌入式代理.
- 为了实现智能行为模式,并改善导航性能,没有环境先验.
主要方法:
- 开发了一个端到端的视觉运动导航框架,集成NeRF用于在线空间内存构建.
- 引入了结构辐射注意 (SRA) 来从NeRF的隐性结构中提取历史见解.
- 在图像目标导航任务上训练并测试了系统,并部署在移动机器人上.
主要成果:
- 基于NeRF的框架在图像目标任务中明显优于现有的导航模型.
- 通过利用NeRF内存,结构辐射注意力 (SRA) 可以明显改善场景和任务的理解.
- 该系统在现实世界机器人导航中展示了增强的探索意识和实时性能.
结论:
- 拟议的端到端框架有效地将NeRF集成到认知导航中,消除了模块化设计的需要.
- SRA增强了代理人掌握复杂场景和任务目标的能力,从而导致智能行为.
- 该系统在现实应用中展示了强大的认知能力和实时性能.
更多相关视频
相关概念视频
01:24Vision
53.1K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.1K
01:14Motor and Sensory Areas of the Cortex
3.6K
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex....
3.6K
01:26Visual System
558
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
558
01:27Diencephalon: Thalamus and Information Relay
1.5K
The thalamus, often called “the gateway to the cerebral cortex,” is vital in processing and directing sensory and motor signals throughout the brain. Almost all inputs destined for the cerebral cortex, except for olfactory signals, are relayed through the thalamus. The thalamus is a sophisticated relay station, channeling information from various brain regions to the cerebral cortex, as well as a filter, prioritizing certain signals over others based on current physiological...
1.5K
01:15Depth Perception and Spatial Vision
610
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
610