Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination

Area of Science:

• Computer Vision
• Computer Graphics
• Material Science

Background:

• Recovering object materials from images is challenging due to inherent ambiguities in inverse rendering.
• Existing methods struggle with accuracy because of the complex coupling between geometry, materials, and lighting.

Purpose of the Study:

• To develop a novel approach for accurate object material recovery from posed images under unknown lighting.
• To overcome the ill-posed nature of inverse rendering by introducing a learned material prior.

Main Methods:

• Utilized differentiable physically based rendering for material parameter optimization.
• Introduced a generative model, specifically diffusion models, to learn priors for albedo and specular properties.
• Developed a coarse-to-fine training strategy for multi-view consistency.

Main Results:

• Achieved state-of-the-art performance in object material recovery on both real-world and synthetic datasets.
• Demonstrated the model's versatility in resolving ambiguities in material representation from RGB images.
• Showcased improved stability and accuracy through the multi-view consistent training strategy.

Conclusions:

• The proposed method effectively recovers object materials by regularizing inverse rendering with learned generative priors.
• The approach offers a robust solution for material estimation, outperforming previous techniques.
• The code is publicly available, facilitating further research in intrinsic image decomposition and material recovery.