Multi-Layer Scene Representation from Composed Focal Stacks.

Area of Science:

• Computer Vision
• Computer Graphics
• Virtual and Augmented Reality

Background:

• Multi-layer images are crucial for high-performance rendering in virtual/augmented reality (VR/AR).
• Current methods using deep neural networks often suffer from local noise in input images, degrading rendering quality and view transition coherence.
• Limited nearest views in training data exacerbate noise issues.

Purpose of the Study:

• To propose and analyze the use of focal stacks for generating noise-robust multi-layer images.
• To enhance the coherence and quality of rendering in VR/AR applications.
• To demonstrate the benefits of focal stacks in terms of memory footprint and data capturing for AR.

Main Methods:

• Utilized focal stacks composed of multi-view inputs to mitigate noise in image data.
• Developed a deep neural network approach for encoding colors and alpha values on each layer.
• Provided theoretical analysis for optimal focal stack configurations.

Main Results:

• Focal stacks significantly diminish noise in multi-layer image generation.
• Demonstrated advantages in coherent rendering, reduced memory footprint, and improved AR-supported data capturing.
• Showcased three distinct applications of the proposed imaging technique for VR.

Conclusions:

• Focal stacks offer a robust solution for generating high-quality multi-layer images for VR/AR.
• The proposed method improves rendering coherence and efficiency.
• The technique has practical implications for AR data capture and VR applications.