Pose2Gaze:日常活动中的眼睛与身体的协调,用于从全身姿势预测眼神
IEEE transactions on visualization and computer graphics
|June 11, 2024
概括
这项研究揭示了眼睛与身体的协调在虚拟和增强现实 (VR/AR) 应用中显著改善了视线预测. 新的Pose2Gaze模型利用全身姿势和头部方向来更准确地估计眼睛的凝视.
科学领域:
- 计算机视觉 计算机视觉
- 人与计算机的交互
- 虚拟现实和增强现实
背景情况:
- 人类眼睛的凝视对于VR/AR应用程序至关重要,例如凝视条件染和交互.
- 以前的凝视预测模型仅专注于眼睛-头部协调,将其范围限制在人-物体交互上.
研究的目的:
- 在各种人与物体和人与人之间的互动中全面分析眼睛与身体的协调.
- 开发和评估一种新的眼睛-身体协调模型 (Pose2Gaze) 以提高视线预测.
主要方法:
- 分析了四个公共数据集 (MoGaze,ADT,GIMO,EgoBody),涵盖了现实世界,VR和AR环境.
- 开发Pose2Gaze,一种利用卷积神经网络和时空图卷积网络来处理头部方向和全身姿势的模型.
- 将Pose2Gaze与仅依靠头部运动的最先进方法进行比较.
主要成果:
- 眼睛的目光与人与物体的互动 (例如,选择和放置) 中的全身运动有很强的相关性.
- 凝视方向与人与人互动 (例如,聊天,教学) 中对交互伙伴的身体方向相关.
- 与基线方法相比,Pose2Gaze显著表现出色,在数据集中平均角度误差降低了10.1%28.6%.
- 此外,Pose2Gaze在基于眼睛的活动识别下游任务中也表现出卓越的表现.
结论:
- 眼睛与身体的协调包含了大量的信息,用于在日常活动中准确地预测眼神.
- "Pose2Gaze"模型通过结合身体姿势信息,在目光预测方面取得了重大进展.
- 这项工作为凝视预测研究开辟了新的途径,特别是在复杂的交互环境中.
相关概念视频
Accessory Structures of the Eye
1.5K
Optical perception, or vision, is an extraordinary sense dependent on converting light signals received via the ocular organs. These organs, known as eyes, are securely positioned within the bony cavities of the skull, called orbits. The orbits serve a dual purpose: a protective shield for the ocular globes and a stable attachment point for the soft ocular tissues. The eye's external protective mechanisms include the eyelids, which are edged with lashes that act as a barrier against foreign...
1.5K
Association Areas of the Cortex
5.3K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.3K
Anatomy of the Eyeball
7.0K
The eye is a spherical, hollow structure composed of three tissue layers. The outer layer — the fibrous tunic, comprises the sclera — a white structure — and the cornea, which is transparent. The sclera encompasses some of the ocular surface, most of which is not visible. However, the 'white of the eye' is distinctively visible in humans compared to other species. The cornea, a clear covering at the front of the eye, enables light penetration. The eye's middle...
7.0K
Relative Motion Analysis using Rotating Axes
459
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
459


