Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Estimating reflectance and illumination from images is crucial for realistic rendering and scene understanding.
• Previous methods often require known object shape, reflectance, or illumination, limiting their applicability.
• A data-driven approach is needed to handle complex natural illumination and single-material specular objects.

Purpose of the Study:

• To develop a method for estimating reflectance and illumination from a single image of a single-material specular object.
• To overcome limitations of prior work by not assuming known object components (shape, reflectance, or illumination).
• To provide a robust, learning-based solution for material and lighting decomposition.

Main Methods:

• A two-step approach using Convolutional Neural Networks (CNNs).
• Step 1: Estimate reflectance map directly from the image or indirectly via surface orientation estimation.
• Step 2: Decompose reflectance map into Phong reflectance parameters and spherical illumination maps using a proposed CNN architecture.

Main Results:

• The proposed CNNs effectively estimate reflectance maps and decompose them into reflectance and illumination.
• New datasets were created to train the CNNs, enabling robust performance on synthetic and real data.
• Demonstrated significant improvements over state-of-the-art methods in quantitative and qualitative evaluations.

Conclusions:

• The developed method successfully estimates reflectance and illumination from single images of specular objects without prior assumptions.
• The learning-based approach with CNNs offers a powerful solution for material and lighting decomposition.
• The method shows broad applicability and outperforms existing techniques, advancing the field of inverse rendering.