VL-HTR:从视觉语言模型学习人与目标的表达
IEEE transactions on cybernetics
|February 18, 2026
概括
这项研究介绍了VL-HTR,这是一种新的视觉语言方法,用于人类的目光目标预测. 它使用多模式知识来提高准确性,并大大加快培训趋同,以更好地理解人类目标.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 现有的人类凝视目标预测方法经常使用仅视觉特征.
- 这些方法与小/被遮蔽的物体作斗争,并且缺乏精确的头部方向先验,导致缓慢的融合和糟糕的性能.
研究的目的:
- 介绍VL-HTR,这是一个新的视觉语言学习方法,用于人类目标的表现.
- 从视觉语言模型 (VLM) 整合多式联络知识,建立强大的人与目标关系.
- 增强语义理解,加快凝视点回归.
主要方法:
- 使用预训练VLM提取多式联运特征,以提高人类目标知识.
- 使用语言引导查询对齐 (LQA) 模块用于语义意识对象表示.
- 使用语言引导方向预测 (LDP) 模块进行多式模式的人类视线方向预测.
主要成果:
- VL-HTR在凝视对象预测 (GOP) 和凝视目标估计任务上表现出卓越的性能.
- 在五个具有挑战性的基准上实现了显著更快的培训融合.
- 在建立人与目标关系方面经过验证的有效性.
结论:
- VL-HTR有效地利用多式联运信息来增强人类目光目标预测.
- 与仅视觉技术相比,拟议的方法提供了一种更强大,更有效的方法.
- VL-HTR对推进眼神分析和人机交互的研究有前途.
相关概念视频
01:24Vision
60.4K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
60.4K
01:10Higher Mental Functions of the Brain: Language
3.9K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.9K
01:27Language and Cognition
836
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
836