High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Graphics

Background:

• Reconstructing 3D human faces from single in-the-wild images is a complex computer vision challenge.
• Existing methods often require multiple images or controlled conditions, limiting their real-world applicability.

Purpose of the Study:

• To propose a novel model-based deep convolutional autoencoder for accurate 3D face reconstruction from a single color image.
• To enable unsupervised, end-to-end training on large, unlabeled datasets.

Main Methods:

• A convolutional neural network (CNN) encoder is combined with a differentiable, parametric, model-based generative decoder.
• The decoder analytically models image formation, encoding semantic parameters like pose, shape, expression, reflectance, and illumination.
• Unsupervised end-to-end training is achieved by integrating the CNN encoder and generative decoder.

Main Results:

• The proposed method achieves state-of-the-art 3D face reconstruction quality and representational richness.
• The CNN encoder effectively learns semantically meaningful parameters from monocular input.
• Introduced stochastic vertex sampling and analysis-by-synthesis/shape-from-shading for enhanced fidelity and faster training.

Conclusions:

• The novel approach successfully integrates model-based and CNN-based techniques for robust 3D face reconstruction.
• Unsupervised training on large datasets is feasible, paving the way for more accessible 3D face modeling.
• The method offers significant improvements over existing approaches for single-image 3D face reconstruction.