3DFaceShop: Explicitly Controllable 3D-Aware Portrait Generation.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Traditional avatar creation is expensive.
• Generative models offer direct learning from photographs.
• Controllable generative models struggle with multi-view consistency, especially for large poses.

Purpose of the Study:

• To propose a network for generating 3D-aware portraits with semantic control over pose, identity, expression, and illumination.
• To address the challenge of multi-view consistency in generative portrait synthesis.
• To improve the realism and controllability of generated 3D portraits.

Main Methods:

• Utilizes neural scene representation for 3D-aware portrait modeling.
• Employs a parametric face model for explicit control.
• Introduces a volume blending strategy for seamless expression animation by segmenting dynamic and static areas from a semantic field.

Main Results:

• Achieves superior performance compared to prior methods in generating realistic portraits.
• Produces vivid expressions under natural lighting conditions with free viewpoint rendering.
• Demonstrates generalization to real images and out-of-domain data.

Conclusions:

• The proposed network effectively generates controllable, 3D-aware portraits with high visual fidelity.
• The volume blending strategy successfully resolves inconsistencies in non-face regions during expression animation.
• The method shows significant promise for real-world applications in digital humans and virtual reality.