3DeepVOG: An Open-Source Framework for Real-Time, Accurate 3D Gaze Tracking with Deep Learning

Eye movements are key biomarkers for diagnosing and monitoring neuro-otological, neuro-ophthalmological and neurodegenerative disorders. Video-oculography (VOG) systems enable detection of small, rapid eye movements and subtle oculomotor pathologies that may be missed during clinical exams. However, they rely on high-quality input, struggle with torsional movements, and are often limited by high costs in clinical and research settings.

To overcome these limitations, we developed 3DeepVOG, a deep learning-based framework for three-dimensional monocular gaze tracking (horizontal, vertical, and torsional rotation) that operates robustly across varied imaging conditions, including low-light and noisy environments. The method combines automated pupil and iris segmentation with geometrically interpretable estimation using a two-sphere anatomical eyeball model with corneal refraction correction. Torsion is tracked in real time using a novel mini-patch template matching approach. The system was trained on over 24,000 annotated samples obtained across multiple devices and clinical scenarios. Application was tested against a gold-standard VOG system in healthy controls.

3DeepVOG operates in real time (>300 fps) and achieves gaze errors of ∼0.1° in all three dimensions. Oculomotor measures - saccadic peak velocity, smooth pursuit gain, and optokinetic nystagmus slow-phase velocity - show good-to-excellent agreement with a clinical gold-standard system. As proof of concept, we present a case of acute unilateral vestibular failure where 3DeepVOG reliably captures 3D nystagmus.

3DeepVOG enables accurate, quantitative eye movement tracking across three dimensions under diverse conditions. As an open-source framework, it provides an accessible and scalable tool for advancing research and clinical assessment in neurological oculomotor disorders.