WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Dynamic human avatar reconstruction is crucial for applications like virtual reality and animation.
• Existing methods often require full body coverage, limiting their use with common monocular videos.
• Reconstructing unseen body parts from limited views remains a significant challenge.

Purpose of the Study:

• To introduce WonderHuman, a novel method for high-fidelity dynamic human avatar reconstruction from monocular videos.
• To enable accurate rendering of unseen human body parts using generative model priors.
• To achieve photorealistic novel view synthesis of dynamic human subjects.

Main Methods:

• Leveraging 2D generative diffusion model priors for high-quality reconstruction.
• Implementing Dual-Space Optimization with Score Distillation Sampling (SDS) in canonical and observation spaces.
• Employing a View Selection strategy and Pose Feature Injection for data consistency and fidelity.

Main Results:

• Achieved state-of-the-art (SOTA) performance in dynamic human avatar reconstruction from monocular videos.
• Demonstrated high-fidelity, photorealistic rendering, especially for challenging unseen body parts.
• Successfully reconstructed dynamic avatars with accurate pose-dependent effects.

Conclusions:

• WonderHuman effectively reconstructs dynamic human avatars from limited monocular video input.
• The proposed method overcomes previous limitations by accurately rendering unseen body parts.
• This work advances the field of dynamic avatar creation for realistic virtual experiences.