CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generation

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Ego-centric 3D scene generation faces challenges with limited view overlap and perspective bias.
• Existing methods struggle with viewpoint consistency, semantic alignment, and geometric accuracy.

Purpose of the Study:

• To propose CGGS, a novel text-to-3D framework for enhanced ego-centric 3D scene generation.
• To address limitations in 3D content awareness and geometric distortions.
• To generate coherent and accurate text-driven 3D scenes.

Main Methods:

• Ego-centric Generator: Fine-tuned Multi-View Latent Diffusion Model with neural-augmented loss for consistent 2D content generation.
• Layout Decorator: Utilized optical flow and point-track correspondence for depth estimation and coarse point cloud layout generation.
• Geometric Refiner: Employed entropy-based Mutual Information Depth Loss (MID) and hierarchical optimization for 3D Gaussian reconstruction refinement.

Main Results:

• CGGS successfully generates consistent, high-fidelity 2D content aligned with textual descriptions.
• The framework produces dense point clouds as coarse layouts from 2D priors.
• Enhanced 3D Gaussian reconstruction shows improved visual quality and geometric structure.

Conclusions:

• CGGS effectively enhances 3D content awareness and mitigates geometric distortions in ego-centric scene generation.
• The proposed framework outperforms previous methods in creating coherent and accurate text-driven 3D scenes.
• CGGS represents a significant advancement in text-to-3D generation for ego-centric perspectives.