TripoSG: High-Fidelity 3D Shape Synthesis Using Large-Scale Rectified Flow Models.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• Diffusion models have advanced 2D image and video generation.
• 3D shape generation lags due to data scale and processing challenges.
• Existing 3D methods struggle with quality, generalization, and input alignment.

Purpose of the Study:

• Introduce TripoSG, a novel diffusion paradigm for high-fidelity 3D mesh generation.
• Improve 3D shape generation quality, generalization, and input correspondence.
• Address limitations in current 3D generative models.

Main Methods:

• Developed a large-scale rectified flow transformer for 3D shape generation.
• Implemented a hybrid supervised training strategy using SDF, normal, and eikonal losses for 3D VAE.
• Created a data processing pipeline generating 2 million high-quality 3D samples.

Main Results:

• Achieved state-of-the-art fidelity in 3D shape generation.
• Generated high-resolution 3D meshes with precise correspondence to input images.
• Demonstrated improved versatility and generalization across diverse image inputs.

Conclusions:

• TripoSG's components effectively enhance 3D generative model performance.
• The framework offers a significant advancement in generating detailed and accurate 3D shapes.
• This work provides a foundation for future innovation in 3D generation.