See360: Novel Panoramic View Interpolation

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Existing view rendering methods often limited to indoor/synthetic scenes or small objects.
• Lack of efficient methods for 360° panoramic view interpolation and camera-centered view synthesis.

Purpose of the Study:

• Introduce See360, a versatile framework for 360° panoramic view interpolation.
• Enable effective camera-centered view synthesis and scene exploration without depth maps or point clouds.

Main Methods:

• Utilize latent space viewpoint estimation for novel view synthesis.
• Employ a Multi-Scale Affine Transformer (MSAT) for coarse-to-fine feature rendering.
• Implement a Conditional Latent space AutoEncoder (C-LAE) for arbitrary angle view interpolation.

Main Results:

• Achieved real-time rendering of arbitrary views across four diverse datasets (UrbanCity360, Archinterior360, HungHom360, Lab360).
• Demonstrated versatility by successfully rendering unknown real-world scenes with minimal additional training.
• Showcased superior performance in 360° panoramic view interpolation and camera-centered view rendering.

Conclusions:

• See360 provides an efficient and versatile solution for 360° panoramic view interpolation.
• The method's ability to generalize to real-world scenes opens new possibilities for immersive visual experiences.
• See360 advances camera-centered view rendering and scene exploration capabilities.