Probability-Based Rendering for View Synthesis | JoVE Visualize

Area of Science:

Computer Vision
Computer Graphics
Image Processing

Background:

Traditional depth image-based rendering (DIBR) synthesizes intermediate views using explicit geometric information, which is prone to rendering artifacts due to depth estimation errors.
Existing view synthesis methods often struggle with accuracy and consistency, especially when dealing with complex scenes or minor inaccuracies in depth maps.

Purpose of the Study:

To introduce a robust probability-based rendering (PBR) method for intermediate view synthesis.
To enhance view synthesis by addressing the limitations of conventional DIBR techniques, particularly their sensitivity to depth errors.
To develop a method that produces visually consistent and artifact-free synthesized views.

Main Methods:

Formulating view synthesis as an image fusion process, adaptively blending textures based on steady-state matching probability (SSMP).
Inferring SSMP for each pixel using the random walk with restart (RWR) algorithm, ensuring visually consistent matching probabilities.
Leveraging RWR for its parameter independence, unlike conventional optimization schemes that are sensitive to parameter choices.

Main Results:

The PBR method demonstrates superior robustness against depth estimation errors compared to existing view synthesis approaches.
Experimental results show significant qualitative and quantitative improvements, effectively suppressing flicker artifacts in virtual video rendering.
The depth maps derived from the RWR-based SSMP calculation are comparable to state-of-the-art local stereo matching methods.

Conclusions:

The proposed PBR method offers a more robust and artifact-resistant solution for intermediate view synthesis.
The RWR algorithm provides a reliable mechanism for inferring matching probabilities, leading to consistent and accurate results.
The PBR method shows potential for applications in virtual reality, augmented reality, and 3D content creation, even outperforming current stereo matching techniques in depth map generation.