GANG: Geometrically-Aligned Neural Gaussians for Efficient and Realistic Relighting

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Rendering

Background:

• Efficient and realistic relighting of complex scenes with unknown illumination is a significant challenge.
• Existing 3D Gaussian Splatting (3DGS) methods show promise but struggle with complex scenes due to difficulties in decoupling geometry, materials, and lighting.

Purpose of the Study:

• To introduce a novel method, Geometrically-Aligned Neural Gaussian Splatting (GANG), for efficient and physically based rendering (PBR) on relightable neural Gaussians.
• To achieve accurate decoupling of intricate scene properties for realistic relighting.

Main Methods:

• GANG regularizes neural Gaussians using a differentiable implicit indicator function (IIF) solver for geometric alignment with a latent signed distance field (SDF) surface.
• A locally consistent geometry regularization and a hybrid lighting model (spherical Gaussians and environment map) are employed for concise neural Gaussian learning and accurate illumination modeling.

Main Results:

• GANG demonstrates superior performance in material decomposition and relighting quality compared to previous PBR methods on public datasets.
• The method effectively represents complex scenes using concise anchors, achieving state-of-the-art results in realistic relighting.

Conclusions:

• GANG represents a significant advancement in 3DGS for realistic relighting, offering efficient rendering and flexible editing of materials and illumination.
• The proposed approach effectively addresses the challenges of complex scene relighting, setting a new state-of-the-art.