ReenactArtFace: Artistic Face Image Reenactment.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Deep generative models have advanced human face reenactment using real images.
• Existing methods struggle with artistic faces due to domain gaps, failing to preserve unique characteristics like identity and contour lines.

Purpose of the Study:

• To introduce ReenactArtFace, the first solution for transferring poses and expressions from human videos to diverse artistic face images.
• To address the limitations of current face reenactment techniques when applied to artistic domains.

Main Methods:

• A coarse-to-fine approach involving 3D artistic face reconstruction using a 3D morphable model (3DMM) and a 2D parsing map.
• Artistic face refinement using a personalized conditional adversarial generative model (cGAN) fine-tuned on input data.
• Introduction of a novel contour loss to ensure faithful synthesis of contour lines during refinement.

Main Results:

• Successfully reconstructs textured 3D artistic faces, enabling robust coarse reenactment under varying poses and expressions.
• Refines coarse results to overcome self-occlusions and synthesize accurate contour lines, preserving artistic face characteristics.
• Demonstrates superior performance over existing solutions in both quantitative and qualitative evaluations.

Conclusions:

• ReenactArtFace effectively bridges the domain gap between real and artistic faces for reenactment tasks.
• The proposed coarse-to-fine strategy with 3DMM and cGAN, enhanced by contour loss, achieves high-fidelity artistic face reenactment.
• This work opens new possibilities for manipulating and animating artistic representations of faces.