FreeGaze: A Framework for 3D Gaze Estimation Using Appearance Cues from a Facial Video

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Behavioral Neuroscience

Background:

• Gaze behavior is a key indicator of human attention.
• Estimating gaze from facial videos is challenging due to variations in appearance and head pose.
• Accurate gaze estimation has applications in various fields, including user experience research and assistive technologies.

Purpose of the Study:

• To develop a robust framework for 3D gaze estimation using appearance cues.
• To improve the accuracy and computational efficiency of gaze estimation methods.
• To introduce and evaluate a novel dual-branch convolutional neural network (FG-Net) for this task.

Main Methods:

• An end-to-end facial landmark detection approach.
• An improved normalization method using orthogonal matrices for enhanced accuracy and reduced computational time.
• A dual-branch convolutional neural network (FG-Net) integrating eye and face features for 3D gaze vector estimation.

Main Results:

• The improved normalization method demonstrated higher accuracy and lower computational time compared to existing methods.
• FG-Net achieved remarkable accuracies of 3.11° on the MPIIGaze dataset and 2.75° on the EyeDiap dataset.
• Ten-fold cross-validation confirmed the framework's effectiveness and state-of-the-art performance.

Conclusions:

• The proposed framework offers a highly effective solution for 3D gaze estimation.
• FG-Net represents a significant advancement in accurately estimating gaze direction from facial videos.
• The study highlights the potential of appearance-based cues and deep learning for sophisticated gaze analysis.