HuGDiffusion: Generalizable Single-Image Human Rendering via 3D Gaussian Diffusion.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Novel view synthesis (NVS) of human characters often requires multiple calibrated images or videos.
• Real-world scenarios frequently lack such controlled input data, limiting existing methods.
• Generating 3D representations from single images presents a significant challenge.

Purpose of the Study:

• To develop a generalizable pipeline for novel view synthesis of human characters from single-view images.
• To leverage diffusion models for generating 3D Gaussian Splatting (3DGS) attributes conditioned on human priors.
• To overcome the limitations of existing NVS methods that rely on multi-view or video inputs.

Main Methods:

• A diffusion-based framework is employed to generate 3DGS attributes from single human images.
• Human-centric feature extraction is used to derive conditioning signals for the diffusion model.
• A multi-stage generation strategy is designed to optimize the learning of different 3DGS attributes.
• Proxy ground-truth 3D Gaussian attributes are constructed to provide high-quality supervision signals.

Main Results:

• HuGDiffusion demonstrates significant performance improvements over state-of-the-art methods for human character NVS.
• The method successfully synthesizes novel views from single input images, even with unknown camera poses.
• The pipeline shows generalizability across different human characters and poses.

Conclusions:

• HuGDiffusion provides an effective solution for novel view synthesis of human characters using single-view inputs.
• The proposed diffusion-based approach with multi-stage attribute generation advances the field of 3D character reconstruction.
• The publicly available code facilitates further research and application in 3D human modeling and rendering.