Photorealistic Monocular Gaze Redirection Using Machine Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Image Synthesis

Background:

• Gaze redirection is crucial for applications like virtual reality and teleconferencing.
• Existing methods often require multiple cameras or struggle with real-time performance and generalization.

Purpose of the Study:

• To develop a general, learning-based approach for monocular gaze redirection.
• To create systems that can realistically alter gaze direction in images while generalizing to new subjects and conditions.

Main Methods:

• Training image re-synthesis models on pairs of images with known gaze disparities.
• Developing three distinct system instantiations: decision forest-based (real-time, fixed angle), deep architecture-based (photorealistic, variable angles), and a hybrid (real-time decision forests with deep network supervision).

Main Results:

• Learning-based re-synthesis successfully achieves convincing gaze redirection from monocular input.
• The hybrid system demonstrates high photorealism and real-time performance, approaching the quality of a deep network teacher.
• All systems show good generalization to unseen people and imaging conditions.

Conclusions:

• Machine learning offers a powerful solution to the monocular gaze redirection problem.
• The proposed hybrid system provides a highly realistic and efficient real-time solution for gaze correction.