Learning Residual Color for Novel View Synthesis.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Scene Representation Networks (SRNs) map spatial coordinates to color and density but struggle with high-frequency details, causing blurriness in novel views.
• Existing methods using fully connected networks often fail to memorize complex scene textures due to limited parameters.

Purpose of the Study:

• To improve novel view synthesis by addressing the limitation of SRNs in rendering high-resolution details.
• To propose a new method that learns 'residual color' to overcome the memorization challenges of complex scene textures.

Main Methods:

• The proposed method learns the 'residual color'—the difference between surface color and a reference color derived from spatial color priors.
• Reference colors are extracted from input view observations, providing a basis for calculating residuals.
• A novel view synthesis system utilizes Scene Representation Networks (SRNs) to learn these residual colors.

Main Results:

• The approach demonstrates competitive performance in preserving high-resolution details during novel view synthesis.
• Rendering results show visually more pleasant outputs compared to current state-of-the-art methods.
• Learning residual color proved more effective for networks with limited parameters than learning radiance color directly.

Conclusions:

• Learning residual color is a more effective strategy for Scene Representation Networks (SRNs) to handle high-frequency details in novel view synthesis.
• The proposed method offers a significant improvement in visual quality for synthesized novel views, reducing blurriness and preserving details.
• This work advances the field of novel view synthesis by providing a robust solution for capturing intricate scene textures.