3DeepVOG: An Open-Source Framework for Real-Time, Accurate 3D Gaze Tracking with Deep Learning

Area of Science:

• Neuroscience
• Ophthalmology
• Medical Technology

Background:

• Eye movements are crucial biomarkers for neurological disorders.
• Traditional video-oculography (VOG) systems have limitations in accuracy, torsional movement tracking, and cost.
• Existing VOG systems require high-quality imaging and struggle in challenging conditions.

Purpose of the Study:

• To develop a deep learning framework for accurate 3D eye movement tracking.
• To overcome the limitations of current VOG systems, including robustness in varied imaging conditions and cost-effectiveness.
• To enable real-time tracking of horizontal, vertical, and torsional eye movements.

Main Methods:

• Developed 3DeepVOG, a deep learning framework for 3D monocular gaze tracking.
• Integrated automated pupil/iris segmentation with a two-sphere eyeball model and corneal refraction correction.
• Employed a novel mini-patch template matching for real-time torsional tracking.

Main Results:

• 3DeepVOG achieves real-time performance (>300 fps) with high accuracy (∼0.1° gaze error) in all three dimensions.
• Oculomotor measures demonstrated good-to-excellent agreement with a gold-standard VOG system.
• Successfully captured 3D nystagmus in a case of acute unilateral vestibular failure.

Conclusions:

• 3DeepVOG provides accurate, quantitative 3D eye movement tracking across diverse conditions.
• The open-source framework offers an accessible and scalable tool for neurological oculomotor disorder research and clinical assessment.