RGBE-Gaze: A Large-Scale Event-Based Multimodal Dataset for High Frequency Remote Gaze Tracking

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Biomedical Engineering

Background:

• High-frequency gaze tracking is crucial for applications like foveated rendering and medical diagnosis.
• Current systems face limitations in frequency, power consumption, and comfort, especially in remote scenarios.
• Existing event-camera datasets are limited to near-eye applications, neglecting remote tracking benefits.

Purpose of the Study:

• To introduce the first large-scale, multimodal dataset (RGBE-Gaze) for high-frequency remote gaze tracking.
• To develop and evaluate a hybrid frame-event gaze estimation method tailored for this dataset.
• To demonstrate the benefits of incorporating event streams into remote gaze tracking systems.

Main Methods:

• Collected a multimodal dataset from 66 participants, synchronizing RGB and event cameras.
• Captured 3.6 million RGB images, 26.3 billion event samples, and precise gaze references.
• Developed a hybrid frame-event gaze estimation approach and evaluated benchmarking methods.

Main Results:

• The RGBE-Gaze dataset provides comprehensive data including head pose, gaze direction, and pupil size.
• The proposed hybrid method shows improved gaze tracking frequency and robustness.
• Event stream integration significantly enhances estimation performance across various factors.

Conclusions:

• RGBE-Gaze is a valuable resource for advancing high-frequency remote gaze tracking research.
• Hybrid frame-event methods offer a promising direction for more accurate and robust gaze estimation.
• Event cameras unlock new possibilities for comfortable and effective remote eye-tracking applications.