Physically Based Facial Texture Generation in the Wild.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• 3D facial texture generation is crucial for realistic rendering.
• Existing methods often require controlled environments and specialized hardware (e.g., Light Stage).
• Compatibility with the physically based rendering (PBR) pipeline is a significant challenge.

Purpose of the Study:

• To develop a text-driven framework for generating PBR-compatible 3D facial textures.
• To enable texture generation from images captured in unconstrained, real-world settings.
• To eliminate the dependency on expensive, controlled capture environments.

Main Methods:

• A multistage framework utilizing FFHQUV for texture space normalization.
• Integration of differentiable rendering and texture disentanglement regularization.
• Training a generative adversarial network (GAN) for PBR texture sampling.
• Aligning the GAN's latent space with text embeddings for text-guided generation.
• Designing an edge-aware Score Distillation Sampling (EASDS) loss and boosting scheme.

Main Results:

• The proposed method successfully generates PBR-compatible 3D facial textures from text.
• It effectively handles images captured in unconstrained environments, removing background and hair influences.
• The text-guided generation is flexible and achieves diverse results.
• EASDS improves generation diversity and optimization efficiency.

Conclusions:

• The framework provides a robust solution for text-driven, in-the-wild 3D facial texture generation.
• It overcomes limitations of existing methods regarding PBR compatibility and capture environment requirements.
• The approach demonstrates superior performance compared to current PBR texture generation techniques.