Interactive Rendering of Relightable and Animatable Gaussian Avatars.

Area of Science:

• Computer Graphics
• Computer Vision
• Digital Humans

Background:

• Creating relightable and animatable digital avatars from videos is crucial for virtual reality and digital human applications.
• Existing methods using neural radiance fields or ray tracing suffer from slow training and rendering times.

Purpose of the Study:

• To propose a simple, efficient method for decoupling body materials and lighting from sparse-view or monocular avatar videos.
• To enable interactive frame-rate rendering of avatars under novel viewpoints, poses, and lighting conditions.

Main Methods:

• Utilizing Gaussian Splatting to represent avatar geometry and appearance.
• Obtaining a canonical body mesh via a signed distance function and interpolating attributes to Gaussian kernels.
• Deforming Gaussians to a posed space using forward skinning and incorporating learnable environment lighting for shading.
• Implementing fast shadow modeling through rasterization of the posed body mesh.

Main Results:

• Achieved interactive frame rates (6.9 fps) for rendering animatable and relightable avatars.
• Demonstrated higher quality results compared to previous methods on both synthetic and real datasets.
• Showcased the ability to render avatars under novel viewpoints, poses, and dynamic environmental lighting.

Conclusions:

• The proposed Gaussian Splatting-based method offers a significant improvement in speed and quality for digital avatar creation.
• This approach facilitates real-time rendering and animation of digital humans, advancing virtual reality applications.
• The decoupling of materials and lighting allows for flexible control over avatar appearance under various conditions.