GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Photometric stereo is crucial for 3D shape reconstruction.
• Existing methods often struggle with arbitrary lighting and material variations.
• End-to-end learning offers potential for improved performance.

Purpose of the Study:

• To propose GR-PSN, a novel cascaded framework for surface normal learning and photometric image generation.
• To enable accurate 3D shape reconstruction and arbitrary image rendering from photometric stereo data.
• To outperform existing single-network approaches in surface recovery and material rendering.

Main Methods:

• A cascaded framework comprising GeometryNet and ReconstructNet for end-to-end shape reconstruction and image rendering.
• ReconstructNet incorporates additional supervision for surface-normal recovery, creating a closed-loop with GeometryNet.
• Training utilizes surface-normal loss, reconstruction loss, and transform loss, with stable training achieved through alternate input of predicted and ground-truth normal maps.

Main Results:

• Accurate recovery of object surface normals from an arbitrary number of input images.
• Generation of realistic photometric images with arbitrary surface materials and lighting conditions.
• Demonstrated superiority over single surface recovery networks, achieving realistic rendering across 100 diverse materials.

Conclusions:

• GR-PSN effectively learns surface normals and enables high-quality, arbitrary rendering from photometric stereo.
• The proposed cascaded network architecture and training strategy lead to robust and accurate shape reconstruction and material rendering.
• This method advances the capabilities of photometric stereo in handling complex lighting and material properties.