Dynamic View Synthesis From Small Camera Motion Videos.

Area of Science:

• Computer Vision
• 3D Graphics
• Machine Learning

Background:

• Novel view synthesis for dynamic 3D scenes is challenging, particularly with limited camera motion.
• Existing NeRF-based methods struggle with inaccurate geometry and camera parameters when motion parallax is insufficient.

Purpose of the Study:

• To develop a robust method for novel view synthesis of dynamic 3D scenes with small camera motion.
• To address challenges in scene geometry representation and camera parameter estimation.

Main Methods:

• Proposed Distribution-based Depth Regularization (DDR) to align rendering weight distribution with true distribution.
• Utilized Gumbel-softmax for differentiable sampling and calculated expectation of error.
• Introduced constraints on volume density to ensure correct scene geometry learning.
• Incorporated camera parameter learning during training for enhanced robustness.

Main Results:

• Demonstrated effectiveness in representing scenes with limited camera motion input.
• Achieved favorable comparisons against state-of-the-art methods.
• Proposed a visualization tool for demystifying DDR and observing scene geometry.

Conclusions:

• The proposed DDR method significantly improves novel view synthesis for dynamic 3D scenes with limited camera motion.
• The approach enhances geometric accuracy and camera parameter estimation robustness.
• The method offers a promising solution for challenging view synthesis scenarios.