RENI++: A Rotation-Equivariant, Scale-Invariant, Natural Illumination Prior

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Inverse rendering is an ill-posed problem often relying on shape or appearance priors.
• Existing methods use generic lighting representations (e.g., spherical harmonics) with limited expressivity, especially for specular reflections.
• A need exists for more sophisticated priors on natural illumination to improve inverse rendering.

Purpose of the Study:

• To propose a novel conditional neural field representation for natural illuminations.
• To develop a rotation-equivariant High Dynamic Range (HDR) neural illumination model.
• To enhance the expressivity and accuracy of illumination representation in inverse rendering.

Main Methods:

• Utilized a variational auto-decoder and transformer decoder for conditional neural fields.
• Extended Vector Neurons to incorporate equivariance directly into the architecture.
• Employed a scale-invariant loss function, inspired by depth estimation, for accurate HDR image representation.

Main Results:

• Developed a compact, rotation-equivariant HDR neural illumination model.
• The model captures complex, high-frequency features in natural environment maps.
• Demonstrated applicability in inverse rendering tasks and environment map completion from partial data.

Conclusions:

• The proposed model offers a powerful prior for natural illuminations in inverse rendering.
• Achieved accurate representation of HDR images and complex lighting conditions.
• The work provides a valuable tool for realistic scene reconstruction and relighting.