3DFaceSculptor: A Common Framework for Image-Guided 3D Face Deformation

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Modeling

Background:

• Existing 3D face generation methods rely on neural networks and high-cost datasets, limiting style compatibility.
• These learning-based approaches struggle with diverse face styles not present in training data.

Purpose of the Study:

• To introduce 3DFaceSculptor, a general-purpose, non-training framework for interactive 3D face editing.
• To enable accurate 3D face mesh editing guided by semantic images while preserving topology and rigidity.

Main Methods:

• Utilizes a differentiable renderer to deform source 3D face meshes based on semantic image correspondences.
• Employs a hierarchical optimization architecture to balance global and local shape features.
• Incorporates various strategies and losses to ensure deformation accuracy, surface smoothness, geometric rigidity, and global synchronization.

Main Results:

• 3DFaceSculptor successfully edits 3D face meshes following semantic image guidance.
• The framework demonstrates compatibility with producing various face styles, unconstrained by training datasets.
• Achieves state-of-the-art performance in 3D face editing tasks.

Conclusions:

• 3DFaceSculptor provides a flexible and efficient alternative to training-based 3D face editing methods.
• The proposed hierarchical optimization and loss strategies effectively address challenges in 2D image-guided 3D shape deformation.
• The framework shows significant potential for interactive and versatile 3D face manipulation.