Self-supervised Learning of Detailed 3D Face Reconstruction.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Accurate 3D face reconstruction from single images is challenging.
• Previous methods often rely on supervised learning with ground-truth 3D models.
• A need exists for unsupervised or self-supervised methods for 3D face reconstruction.

Purpose of the Study:

• To present an end-to-end learning framework for detailed 3D face reconstruction from a single image.
• To develop a method that does not require surrogate ground-truth 3D models for supervision.
• To leverage the input image itself as supervision during the learning process.

Main Methods:

• The framework employs a two-stage approach: coarse model regression and detailed displacement map prediction.
• A 3D Morphable Model (3DMM) is used for the coarse representation, combined with a displacement map in UV-space for detail.
• Supervision is achieved through photometric and facial perceptual losses between the input and rendered faces, and an image-to-image translation network predicts the displacement map.

Main Results:

• The proposed method successfully reconstructs detailed 3D faces from single images.
• The framework demonstrates superior performance compared to previous state-of-the-art methods.
• Learning the displacement map in UV-space facilitates explicit face alignment and improves detail learning.

Conclusions:

• The developed end-to-end learning framework offers an effective self-supervised approach for 3D face reconstruction.
• The method's ability to use input images for supervision simplifies the 3D face modeling pipeline.
• This work advances the field by enabling high-fidelity 3D face reconstruction without external ground-truth data.