EX-Gaze:高频和低延迟的凝视跟踪与混合事件框架摄像头用于设备上的扩展现实
IEEE transactions on visualization and computer graphics
|March 10, 2025
概括
使用事件摄像头,EX-Gaze实现了用于扩展现实 (XR) 的高频,实时眼睛跟踪. 这种新的系统为先进的XR应用提供了更高的准确性和较低的延迟.
科学领域:
- 计算机视觉 计算机视觉
- 人与计算机的交互
- 虚拟现实和增强现实
背景情况:
- 眼睛跟踪是扩展现实 (XR) 中的人与计算机交互 (HCI) 的关键模式.
- 新兴的XR应用程序需要高频,实时的眼睛跟踪来实现身份验证和诊断等功能.
- 目前的眼睛追踪解决方案难以满足先进XR的苛刻频率要求.
研究的目的:
- 介绍EX-Gaze,一个基于事件的实时眼睛追踪系统,用于设备上的XR.
- 为了实现高跟踪频率 (2KHz) 与体面的准确性和低延迟.
- 通过高效和高性能的眼睛跟踪来实现先进的XR应用.
主要方法:
- 利用事件摄像头,生物启发的视觉硬件捕捉高时间分辨率事件流.
- 开发了一种轻量级的跟踪框架,用于实时定位学生,并在移动设备上进行跟踪.
- 引入了稀疏事件补丁表示和稀疏事件补丁变压器,以优化事件流的处理.
- 在Jetson Orin Nano上实现了EX-Gaze,优化了GPU和CPU之间的计算调度,用于并行深度神经网络处理.
主要成果:
- 实现了2KHz的高跟踪频率,具有低延迟和不错的准确性.
- 在移动设备 (Jetson Orin Nano) 上,在没有积累延迟的情况下,证明了实时性能.
- 在移动平台上,EX-Gaze在平衡准确性和效率方面超过了其他基于事件的眼睛跟踪方法.
结论:
- EX-Gaze是XR中高频,实时眼睛跟踪的突破性系统.
- 该系统的效率和准确性使其适用于苛刻的XR应用.
- 开发的方法和框架为更复杂的设备上XR体验提供了途径.
相关概念视频
Vision
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
Light Acquisition
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
Depth Perception and Spatial Vision
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.


