Beyond 3DMM: Learning to Capture High-Fidelity 3D Face Shape.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• 3D Morphable Model (3DMM) fitting is crucial for face analysis but often yields low-fidelity 3D faces due to geometric detail loss.
• Existing methods struggle with insufficient ground-truth data, unreliable training, and limited 3DMM representation power.

Purpose of the Study:

• To develop a robust solution for personalized 3D face shape reconstruction with high visual fidelity.
• To overcome limitations in current 3D face reconstruction techniques by capturing fine-grained geometry.

Main Methods:

• Virtually rendering input 2D images in calibrated views to normalize pose and preserve geometry.
• Employing a many-to-one hourglass network for feature fusion and generating vertex displacements for fine geometry.
• Training the network by optimizing visual similarity between rendered multiview images of 3D shapes.
• Generating synthetic ground-truth 3D shapes via RGB-D image registration and data augmentation.

Main Results:

• Achieved superior reconstruction accuracy in face shape compared to existing methods.
• Demonstrated enhanced visual verisimilitude in reconstructed 3D faces.
• Validated the effectiveness of the proposed method across challenging experimental protocols.

Conclusions:

• The proposed method significantly improves 3D face reconstruction accuracy and visual realism.
• The approach effectively addresses the limitations of traditional 3DMM fitting by incorporating fine-grained geometric details.
• This work provides a comprehensive solution for generating highly personalized and accurate 3D face shapes from 2D images.