以示例进行查询:使用基于LLM的场景图表表征的语义交通场景检索.
Yafu Tian1,2,3, Alexander Carballo3,4,5, Ruifeng Li2
1Graduate School of Informatics, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan.
Sensors (Basel, Switzerland)
|April 26, 2025
概括
本研究引入了一种新的方法,用于在自动驾驶中检索交通场景,使用视觉大语言模型 (VLM) 生成的道路场景图 (RSG). 这种方法增强了语义理解,并支持各种查询类型,以改善场景检索.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器人技术 机器人技术 机器人技术
背景情况:
- 自动驾驶系统需要从庞大的数据集中有效地检索特定的交通场景.
- 现有的方法面临的挑战是语义复杂性和用户在交通场景数据的各种需求.
- 自主驾驶数据的异质性和规模造成了重大检索障碍.
研究的目的:
- 为自动驾驶提出一种新的交通场景检索方法.
- 为了利用视觉大语言模型 (VLMs) 进行结构化的道路场景图 (RSG) 表示.
- 为了实现交通场景的灵活和含义丰富的查询.
主要方法:
- 利用VLMs从视频数据中生成结构化的RSG,捕获对象关系和语义属性.
- 开发了一组可扩展的场景属性和基于图形的描述,用于量化场景相似性.
- 创建了RSG-LLM基准数据集,包含1000个交通场景,描述和RSG用于LLM评估.
主要成果:
- 拟议的方法有效地从大型数据库中检索语义上类似的交通场景.
- 该方法支持各种查询格式,包括自然语言,图像,视频剪辑和rosbag文件.
- 证明了VLM在为交通场景生成准确的RSG方面的能力.
结论:
- 由VLM生成的RSG方法为交通场景检索提供了一个全面而灵活的框架.
- 这种方法显著提高了自动驾驶中场景检索的效率和准确性.
- 拟议的框架有助于在自动驾驶系统中的更广泛应用.
更多相关视频
04:48Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.6K
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.6K
相关概念视频
01:42Schemas
11.5K
A schema is a mental construct consisting of a cluster or collection of related concepts (Bartlett, 1932). There are many different types of schemata, and they all have one thing in common: schemata are a method of organizing information that allows the brain to work more efficiently. When a schema is activated, the brain makes immediate assumptions about the person or object being observed.
11.5K
01:24Vision
52.2K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
52.2K
01:23Storage
46
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
46