TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• 3D face reconstruction from single images is challenging due to perceptual sensitivity to fine details.
• Existing methods often lose high-frequency texture information, resulting in low-fidelity approximations.
• Minimizing reconstruction error in low-dimensional spaces can lead to perceptually insignificant yet numerically minute errors.

Purpose of the Study:

• To develop a high-fidelity 3D facial reconstruction method from single images.
• To enhance the recovery of detailed facial geometry and appearance, including textures.
• To address the loss of high-frequency texture details in current reconstruction techniques.

Main Methods:

• Introduced TED-Face, a unified deep neural network for estimating shape, appearance, illumination, camera, dense radiance residuals, and sparse facial texture features.
• Trained the network on multiple popular face reconstruction datasets.
• Developed new metrics: visual fidelity (VIF) and structural similarity (SSIM) to better evaluate perceptual quality.

Main Results:

• Achieved state-of-the-art performance on the FaceWarehouse benchmark with VIF score 0.4802 and SSIM score 0.9622.
• Demonstrated improved VIF and SSIM scores on the LS3D-300W dataset for both indoor and outdoor images compared to Deep3D.
• The method successfully recovers visually more realistic facial appearance details.

Conclusions:

• TED-Face significantly improves 3D facial reconstruction quality from single images.
• The proposed method effectively recovers high-fidelity geometry and enhanced textures.
• The new metrics (VIF and SSIM) provide a better assessment of perceptual realism in face reconstruction.