Adaptive 3D Face Reconstruction from Unconstrained Photo Collections

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• Reconstructing 3D face models from unconstrained images is challenging due to variations in pose, expression, and illumination.
• Existing methods often require large, high-quality photo collections, limiting their applicability.

Purpose of the Study:

• To develop a robust method for 3D face surface and albedo reconstruction from diverse, unconstrained photo collections.
• To adapt reconstruction techniques for varying numbers and qualities of input images.

Main Methods:

• Incorporated prior face shape knowledge by fitting a 3D morphable model to create a personalized template.
• Employed a novel photometric stereo formulation within a coarse-to-fine scheme to refine fine details.
• Utilized a structural similarity-based local selection to identify common expressions and handle occlusions.

Main Results:

• Successfully reconstructed detailed 3D face models and albedo information from diverse image sets.
• Demonstrated adaptability to varying image collection sizes and quality.
• Achieved superior performance on synthetic, internet, and personal photo collections, validated by a novel quality measure.

Conclusions:

• The proposed method offers an effective approach for 3D face reconstruction from challenging, unconstrained image data.
• The technique is adaptable and robust, overcoming limitations of previous methods requiring extensive datasets.
• The novel evaluation metric provides a reliable assessment in the absence of ground truth scans.