Food3D: Text-Driven Customizable 3D Food Generation With Gaussian Splatting

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Realistic 3D food generation is crucial for nutritional assessment, advertising, and virtual content.
• Existing text-to-3D models often suffer from poor initialization and limitations in capturing fine details from text-to-image models, impacting realism.
• A significant gap exists between generated images and real-world visuals, especially for complex food structures.

Purpose of the Study:

• To introduce Food3D, a novel framework for advanced 3D food generation.
• To address limitations in initialization and text-to-image supervision for improved 3D food realism.
• To enhance the visual fidelity, detail, and accuracy of generated 3D food models.

Main Methods:

• Developed Food3D-G, a generalized method using Mamba-based initialization to improve the starting point of 3D generation.
• Introduced Food3D-C, a customizable method employing a dual-branch diffusion model for personalized 3D food generation and intricate detail capture.
• Integrated 3D Gaussian splatting (3D GS) and schedulable interval score matching (S-ISM) for enhanced shape and texture generation in both methods.

Main Results:

• Food3D-G significantly enhances visual fidelity and quality through improved initialization.
• Food3D-C effectively captures intricate details in complex food structures using a dual-branch diffusion model.
• Extensive experiments confirm Food3D achieves state-of-the-art performance with substantial improvements in detail, shape accuracy, and visual realism.

Conclusions:

• The Food3D framework offers a significant advancement in realistic 3D food generation.
• The proposed methods effectively overcome limitations in existing text-to-3D models, particularly for complex food items.
• Food3D provides a robust solution for high-quality, detailed, and visually realistic 3D food models.