GiganticNVS: Gigapixel Large-Scale Neural Rendering With Implicit Meta-Deformed Manifold

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Reconstruction

Background:

• Gigapixel imaging captures abundant scene details but is underutilized in 3D reconstruction.
• Existing novel view synthesis (NVS) methods struggle with large-baseline challenges and high-resolution exploitation, leading to blurred artifacts.

Purpose of the Study:

• To bridge the gap between gigapixel imaging capabilities and 3D reconstruction by addressing the large-baseline problem.
• To introduce GiganticNVS, a novel method for gigapixel large-scale novel view synthesis (NVS).

Main Methods:

• Proposed a meta-deformed manifold representation for implicit neural fields, embedding geometry and appearance into a high-dimensional latent space.
• Utilized featuremetric deformation to enforce multi-view geometric correspondence and learned the reflectance field on the surface.
• Leveraged a highly-expressive implicit field with view-consistency for synthesizing high-fidelity details from large-baseline observations.

Main Results:

• GiganticNVS outperforms state-of-the-art methods quantitatively and qualitatively on standard and gigapixel-level ultra-large-scale benchmarks.
• The method successfully synthesizes high-fidelity details from large-baseline observations, overcoming limitations of existing NVS techniques.
• Demonstrated effective recovery of faithful underlying geometry and exploitation of image resolution.

Conclusions:

• The proposed meta-deformed manifold representation is critical for high-fidelity NVS from large-baseline observations.
• GiganticNVS effectively addresses the challenges of large-scale scene reconstruction using gigapixel imagery.
• The method shows significant improvements in novel view synthesis for complex, large-scale environments.