Deep Scene Synthesis of Atlanta-World Interiors from a Single Omnidirectional Image

Area of Science:

• Computer Vision
• Computer Graphics
• Virtual Reality

Background:

• Estimating 3D geometry from single images is challenging due to inherent ambiguities.
• Virtual reality (VR) applications require immersive 3D environments and novel view synthesis.

Purpose of the Study:

• To develop a data-driven approach for extracting geometric and structural information from single panoramic images of interior scenes.
• To enhance 3D immersion in VR applications by rendering scenes from novel viewpoints.

Main Methods:

• A novel end-to-end deep learning approach jointly estimates depth and room structure, leveraging the 'Atlanta-world' prior (horizontal floors/ceilings, vertical walls).
• The method utilizes domain-specific loss functions and extensive training on synthetic panoramic data.
• A lightweight network infers novel panoramic views from translated positions, enabling interactive rates for perspective view generation and upsampling.

Main Results:

• The approach automatically generates new poses around the original camera at interactive rates.
• It produces depth cues suitable for VR applications, particularly with head-mounted displays.
• Extracted floor plans and 3D wall structures support room exploration.

Conclusions:

• The method achieves low-latency performance and improves prediction accuracy over state-of-the-art solutions on indoor panoramic benchmarks.
• It effectively addresses single-image geometry estimation and novel view synthesis challenges for VR.