T2TD: Text-3D Generation Model Based on Prior Knowledge Guidance.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• 3D models are crucial for applications like autonomous driving, VR, and AR.
• A significant data scarcity exists for 3D models, hindering practical applications.
• Generating 3D models from text descriptions is a promising solution but remains challenging.

Purpose of the Study:

• To propose a novel text-to-3D generation model (T2TD) that creates 3D models from textual descriptions.
• To simulate human imaginative mechanisms by leveraging experiential knowledge for 3D model creation.
• To address the limitations of current text-to-3D generation methods by enhancing efficiency and quality.

Main Methods:

• Introduced a text-3D knowledge graph to link 3D models with semantic textual information, mimicking human experiential knowledge.
• Developed a causal inference model to filter relevant structural information from related shapes, discarding irrelevant data.
• Employed a multi-layer transformer architecture for progressive fusion of textual and structural information to improve 3D generation.

Main Results:

• The proposed T2TD model significantly enhances the quality of generated 3D models.
• Experimental results demonstrate superior performance compared to state-of-the-art methods on text2shape datasets.
• The model effectively compensates for structural information deficits in text-to-3D generation.

Conclusions:

• The T2TD model offers an effective approach for generating high-quality 3D models from textual descriptions.
• Leveraging human-like imaginative processes and knowledge graphs improves text-to-3D generation.
• This work advances the field of 3D content creation from natural language inputs.