驱动LLaVA:通过视觉语言模型的人类层面的行为决策
Rui Zhao1, Qirui Yuan1, Jinyu Li1
1College of Automotive Engineering, Jilin University, Changchun 130025, China.
Sensors (Basel, Switzerland)
|July 13, 2024
概括
本研究介绍了DriveLLaVA,一种使用视觉语言模型 (VLMs) 进行自动驾驶行为决策的新方法. 驱动LLaVA在复杂的场景中提高了概括性,超越了基于规则的限制.
科学领域:
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
- 计算机视觉 计算机视觉
背景情况:
- 自动驾驶旨在实现人类水平的性能,行为决策对于在复杂的环境中导航至关重要.
- 当前基于规则的行为决策系统缺乏对未见的驾驶场景的概括能力.
研究的目的:
- 为自动驾驶提出一种新的行为决策方法,该方法利用视觉语言模型 (VLMs) 的概括和常识推理.
- 提高自动驾驶决策过程的透明度和可解释性.
主要方法:
- 开发一个新的指令遵循数据集,包括图像-文本指令和驾驶行为标签.
- 在自定义数据集上使用低等级适应 (LoRA) 技术微调视觉语言模型DriveLLaVA.
- 通过对大规模指令遵循数据集进行广泛实验进行评估.
主要成果:
- 与最先进的方法相比,DriveLLaVA在行为决策方面表现出卓越的表现.
- 该模型有效地处理多样化和复杂的驾驶场景,并展示了强大的强度.
- 这种方法利用VLM的能力来提高自动驾驶的普遍性.
结论:
- 拟议的DriveLLaVA方法在自动驾驶行为决策方面取得了重大进展.
- 视觉驾驶系统为开发更具适应性和普遍性的自动驾驶系统提供了一个有希望的方向.
- 使用LoRA可以有效地优化模型训练并降低计算成本.
更多相关视频
07:09Integrating Visual Psychophysical Assays within a Y-Maze to Isolate the Role that Visual Features Play in Navigational Decisions
Published on: May 2, 2019
6.1K
09:27Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
10.0K
相关概念视频
Nonconscious Mimicry
4.5K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.5K
Language and Cognition
340
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
340
Visual System
565
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
565
Behaviorism
2.3K
The field of behaviorism was pioneered by figures such as Ivan Pavlov, John B. Watson, and B.F. Skinner fundamentally shifted the focus of psychology to the observable and controllable aspects of human and animal behavior. This shift marked a critical evolution in the discipline, emphasizing scientific rigor and experimental methodology.
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
The core premise of behaviorism is its focus on observable behavior rather than internal thoughts or feelings. This approach argues that true scientific...
2.3K
Observational Learning
163
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
163
