A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces

Area of Science:

• Computer Vision
• 3D Reconstruction
• Computer Graphics

Background:

• 3D Morphable Models (3DMMs) require extensive, high-quality training data for accurate face representation.
• Establishing dense, point-to-point correspondence across diverse 3D face scans is crucial for enhancing 3DMM capabilities.
• Current methods often struggle with heterogeneous datasets encompassing varied identities, ethnicities, and expressions.

Purpose of the Study:

• To develop a fully automatic approach for transferring dense semantic annotations across raw 3D faces using a 3DMM.
• To establish accurate dense correspondence between heterogeneous 3D face scans.
• To enhance the descriptive power and generalization capabilities of 3DMMs.

Main Methods:

• A novel formulation learning sparse deformation components with local support on the face.
• An original non-rigid deformation algorithm for precise 3DMM fitting.
• Leveraging a 3DMM to transfer dense semantic annotations to unseen 3D faces.

Main Results:

• The proposed method effectively generalizes to highly diverse 3D face samples.
• Accurate dense correspondence is established even with complex facial expressions.
• A large-scale, heterogeneous 3DMM was built from over 9,000 registered scans, validating the dense registration accuracy.

Conclusions:

• The developed automatic approach accurately establishes dense correspondence for 3D faces.
• This method significantly improves the ability to build comprehensive and descriptive 3D Morphable Models.
• The technique shows strong potential for applications requiring detailed 3D face analysis and synthesis.