在变压器的眼睛:自我中心的凝视估计和超越全球-本地相关性
Bolin Lai1, Miao Liu1,2, Fiona Ryan1
1Georgia Institute of Technology, Atlanta, GA 30308 USA.
概括
这项研究介绍了一种基于变压器的模型,用于对自我中心的目光进行估计,从而提高了对人类意图的理解. 这种新的方法有效地将全球场景背景与本地视觉数据相关联,以准确地预测目光.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 对自我中心的目光估计对于理解日常活动中的人类意图至关重要.
- 现有的方法往往难以将全球场景背景与当地视觉线索相结合.
研究的目的:
- 开发第一个以变压器为基础的模型,用于自我中心的目光估计.
- 为了提高从第一人称视频视角预测人类目光的准确性.
主要方法:
- 一个新的变压器编码器嵌入全球场景背景作为额外的视觉标记.
- 全球-本地相关模块明确模拟全球和本地视觉信息之间的相互作用.
- 在EGTEA Gaze+和Ego4D数据集上进行验证.
主要成果:
- 拟议的模型显著优于以前的最先进的方法.
- 废除研究证实了全球-本地相关模块的有效性.
- 该模型在相关任务上显示出强大的概括能力,例如凝视跳动/固定预测和动作识别.
结论:
- 全球-本地相关性是准确的自我中心的目光预测的关键表示.
- 基于变压器的方法为视线估计研究提供了一个强大的新方向.
- 该模型的性能突显了其在需要理解人类意图的现实应用中的潜力.
相关概念视频
Field Application of Global Positioning System
45
The Global Positioning System (GPS) has become an indispensable tool in fieldwork, offering unparalleled precision and efficiency for surveying, navigation, and infrastructure development. By harnessing signals from a constellation of satellites, GPS receivers determine the location of objects with remarkable speed and accuracy, often completing calculations within a second.Advantages of Modern GPS TechnologyContemporary GPS receivers are designed to meet the practical demands of field...
45
Errors in Global Positioning System
45
Global Positioning System (GPS) technology has revolutionized navigation and positioning, but its accuracy is often compromised by various errors. These errors, stemming from environmental, satellite, and receiver-related factors, require careful mitigation to ensure reliable performance across applications.Atmospheric ErrorsGPS signals travel through the Earth’s ionosphere and troposphere, introducing delays which affect accuracy. The ionosphere is strongly influenced by charged particles,...
45
Types of Global Positioning System Surveys
58
GPS surveying methods vary in application, accuracy, and data collection techniques, catering to diverse surveying and mapping needs. Static GPS, kinematic GPS, and real-time kinematic (RTK) surveying are widely used. Each technique offers distinct advantages.Static GPS involves placing one receiver at a known reference point and another at the target point. It collects exact positional data by observing multiple satellite ranges over an extended period, achieving centimeter-level accuracy for...
58
Geoid and Ellipsoid
35
The Earth's shape is best described as an ellipsoid, a slightly flattened sphere created by rotating an ellipse around its minor axis. This flattening results in the polar axis being about 21 kilometers shorter than the equatorial axis. In contrast, the geoid represents the Earth's gravitational shape and aligns with the mean sea level (MSL). The geoid is an irregular equipotential surface where gravity is perpendicular at every point. Variations in Earth's mass distribution cause geoid...
35
Relative Motion Analysis using Rotating Axes
460
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
460
Non-inertial Frames of Reference
5.9K
A reference frame accelerating or decelerating relative to an inertial frame is a non-inertial frame. To help understand this, consider what taking off in an airplane, turning a corner in a car, riding a merry-go-round, and the circular motion of a tropical cyclone all have in common. All these systems are accelerating, decelerating, or rotating relative to the Earth; hence, they all are non-inertial frames. All these systems exhibit inertial forces, which merely seem to arise from motion,...
5.9K


