3D Gaze Estimation Using RGB-IR Cameras

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Biomedical Engineering

Background:

• Accurate 3D gaze estimation is crucial for understanding user attention in interactive systems.
• Existing methods struggle with pupil detection in corneal images due to reflections.
• Real-time, unobtrusive eye-tracking calibration remains a challenge.

Purpose of the Study:

• To present a robust framework for 3D gaze estimation using corneal imaging.
• To enable reliable pupil tracking and 3D eye model generation in real-time.
• To develop an auto-calibration process that requires no user instruction.

Main Methods:

• A headset with IR and RGB cameras for pupil tracking and corneal imaging.
• Deep learning algorithms for real-time pupil detection and 3D eye model computation.
• A novel approach to transform pupil positions between IR and RGB images for gaze detection.

Main Results:

• Achieved a low 3D gaze error of 2.12 degrees.
• Demonstrated high accuracy in corneal image acquisition with an IoU of 0.71.
• Validated the framework's performance in realistic indoor and outdoor mobile scenarios.

Conclusions:

• The proposed framework offers a reliable and accurate solution for 3D gaze estimation in corneal imaging.
• The auto-calibration method is unobtrusive, simplifying user experience.
• The system shows potential for diverse real-world mobile applications requiring precise gaze tracking.