Controllable Free Viewpoint Video Reconstruction Based on Neural Radiance Fields and Motion Graphs

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Existing free viewpoint video generation methods often lack fine-grained control over pose and body shape.
• Current neural radiance fields (NeRF) approaches may struggle with efficiency and complex motion sequences.

Purpose of the Study:

• To develop a controllable, high-quality free viewpoint video generation method.
• To enable flexible pose and body shape control within generated videos.
• To improve the efficiency of neural radiance field reconstruction for dynamic scenes.

Main Methods:

• Constructing a directed motion graph from captured video sequences for pose parameterization.
• Integrating explicit surface deformation with implicit neural scene representations for shape control.
• Training a local surface-guided neural radiance field (NeRF) for efficient volumetric rendering in local spaces.

Main Results:

• The proposed method achieves realistic free viewpoint video reconstruction.
• It enables user-guided motion traversal via the motion graph.
• Demonstrates plausible body shape control without sacrificing rendering quality.

Conclusions:

• The novel sequence-motion-parameterization strategy enhances reconstruction efficiency by avoiding redundant calculations.
• This work presents the first method supporting both realistic free viewpoint video reconstruction and motion graph-based motion traversal.
• The approach effectively balances control, realism, and efficiency in free viewpoint video generation.