PCGen: A Fully Parallelizable Point Cloud Generative Model

Area of Science:

• Computer Vision
• 3D Graphics
• Machine Learning

Background:

• Real-time computing is essential for 3D extended reality applications.
• Current point cloud generation methods are too slow for real-time use.
• Generative models offer potential for revolutionizing 3D extended reality.

Purpose of the Study:

• Introduce a novel VQVAE model for rapid, high-quality point cloud synthesis.
• Address the real-time computing bottleneck in augmented and virtual reality.
• Provide a compact representation for conditional generation and data exploration.

Main Methods:

• Developed a VQVAE with architectural improvements for robustness, efficiency, and reconstruction quality.
• Incorporated an innovative computation layer, attention mechanism, and error filter in the decoder.
• Introduced a parallel sampling strategy using a double encoding system for quick inference.

Main Results:

• Synthesized high-quality point clouds in milliseconds, significantly faster than existing methods.
• Achieved a compact sample representation suitable for various downstream tasks.
• Demonstrated the effectiveness of individual architectural and sampling contributions.

Conclusions:

• The proposed VQVAE model meets the real-time demands of 3D extended reality.
• The model's compact representation facilitates conditional generation and rapid prototyping.
• This work advances generative models for efficient 3D data synthesis.