KaleidoEye: A Large-scale Dataset and Benchmark for Slippage Robust Gaze Tracking in HMDs

Egocentric gaze tracking is essential for immersive interaction on head-mounted displays (HMDs). However, existing studies typically assume a fixed and tightly fitted headset placement, overlooking the inevitable HMD slippage under real-world conditions. Although several datasets with variable wearing positions have been proposed, the lack of diverse and annotated slippage hinders slippage-robust gaze tracking research. In this paper, we present KaleidoEye, the first large-scale gaze tracking dataset with precisely quantified 6DoF slippage poses. It contains 1.75 million binocular images from 30 subjects, with gaze targets and curated eye-geometry annotations. By covering 11,130 slippage conditions, KaleidoEye captures extensive eye appearance distortions and gaze-space misalignments, enabling comprehensive studies of gaze-tracking under slippage. Based on this dataset, we propose EST-Gaze, a lightweight and slippage-robust gaze-tracking model for resource-constrained devices. By leveraging eye-edge-guided spatial transformations, EST-Gaze learns task-driven input alignment before gaze regression, thereby mitigating slippage-induced image-layout variations and feature-distribution shifts to improve gaze-estimation accuracy. Experimental results demonstrate that KaleidoEye significantly enhances the robustness of existing gaze-tracking models under headset slippage. Moreover, EST-Gaze achieves an average error of 2.53$^\circ$ under slippage without test-time user recalibration, while maintaining real-time performance (40 FPS on HoloLens 2), achieving a favorable trade-off between accuracy and efficiency among the state-of-the-art approaches.