Relightable and Animatable Gaussian Head Avatar From Monocular Videos

Area of Science:

• Computer Vision
• Computer Graphics
• Virtual Reality

Background:

• Accurate relighting is crucial for realistic virtual avatars.
• Creating personalized, relightable avatars from minimal input (monocular video) is challenging due to material ambiguity and view inconsistency.

Purpose of the Study:

• To propose a novel pipeline for generating personalized and relightable avatars from monocular video under unknown lighting conditions.
• To address challenges in material disentanglement and novel-view consistency.

Main Methods:

• Introduced a disentangled dynamic 3D Gaussian representation for diverse material properties and animation.
• Employed a 2D diffusion-based model to predict canonical-lighting images and material maps from casually lit portraits.
• Incorporated a 3D prior to improve novel-view consistency in geometry and appearance.

Main Results:

• Significantly improved reconstruction quality and relighting fidelity.
• Demonstrated the effectiveness of the proposed disentangled 3D Gaussian representation.
• Showcased the utility of diffusion models for resolving material ambiguity.

Conclusions:

• The proposed pipeline offers a practical and cost-effective solution for high-quality personalized avatar creation.
• The method enhances realism and immersion in virtual environments through accurate relighting.
• This work advances the state-of-the-art in monocular avatar reconstruction and relighting.