Building 3D Generative Models from Minimal Data.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Graphics

Background:

• Traditional 3D morphable models (3DMMs) require multiple high-quality 3D scans.
• Existing methods for 3D reconstruction often rely on extensive datasets.
• Inferring 3D structure from 2D images presents a significant challenge.

Purpose of the Study:

• To propose a novel method for constructing generative 3D object models from a single 3D mesh.
• To enhance these models using unsupervised low-shot learning from 2D images.
• To demonstrate the application of this method in the domain of 3D face modeling and recognition.

Main Methods:

• Constructing 3D morphable models (3DMMs) representing shape and albedo using Gaussian processes from a single 3D scan or template.
• Employing unsupervised low-shot learning with a small number of 2D images to refine the 3DMM.
• Utilizing the generative model for inverse graphics (3D reconstruction from 2D) and registration (3D from 3D).

Main Results:

• Successfully generated 3D morphable models from a single 3D template.
• Demonstrated accurate 3D face recognition using only one 3D template per person.
• Showcased the ability to infer 3D reconstructions from 2D images.
• Developed a preliminary unsupervised learning framework for 3D face distribution learning.

Conclusions:

• The proposed method enables the creation of robust 3D generative models from minimal data.
• This approach offers a viable solution for 3D face recognition and reconstruction with limited scans.
• The method provides a potential computational model for understanding infant face perception development.