Deep Photometric Stereo Networks for Determining Surface Normal and Reflectances

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Photometric stereo traditionally relies on simplified reflectance models (e.g., Lambert's model).
• Accurate surface normal and reflectance estimation is crucial for 3D reconstruction and material analysis.
• Designing computationally tractable yet representative reflectance models for complex real-world materials remains a challenge.

Purpose of the Study:

• To develop a novel photometric stereo method utilizing deep learning to overcome limitations of traditional approaches.
• To establish a flexible mapping between complex reflectance observations and surface normals.
• To enable per-pixel estimation of both surface normal and reflectance for enhanced material understanding and scene rendering.

Main Methods:

• A deep neural network, the deep photometric stereo network (DPSN), was designed to infer surface normal and reflectance.
• The network takes reflectance observations under varying light directions as input.
• Training utilized the MERL BRDF dataset to ensure applicability to real-world scenes.

Main Results:

• The proposed DPSN effectively estimates surface normals and reflectances in a per-pixel manner.
• Evaluation on simulated and real-world scenes demonstrated the method's high accuracy.
• The method successfully predicts reflectance, aiding in material characterization.

Conclusions:

• Deep learning offers a powerful approach to photometric stereo, surpassing traditional simplified models.
• The DPSN provides a robust solution for estimating surface properties from image data.
• This method advances the capabilities of 3D reconstruction and material analysis in computer vision and graphics.