Deep spectral reflectance and illuminant estimation from self-interreflections.

Area of Science:

• Computer Vision
• Computational Imaging
• Photometry

Background:

• Estimating spectral properties of surfaces and light from images is crucial for accurate color reproduction and material analysis.
• Interreflections in concave surfaces create RGB gradients that contain valuable information about surface reflectance and illuminant.
• Classical methods struggle with noise and complex interreflection effects.

Purpose of the Study:

• To develop a robust deep learning method for estimating spectral reflectance and spectral power distribution from a single RGB image.
• To leverage physics-based interreflection models within a convolutional neural network (CNN) framework.
• To improve robustness against image noise and outperform existing state-of-the-art methods.

Main Methods:

• A convolutional neural network (CNN) was trained using simulated data generated by a physics-based interreflection model.
• The network learns to interpret RGB gradients caused by interreflections to infer spectral properties.
• The approach couples deep learning with physical interreflection modeling.

Main Results:

• The proposed CNN-based method accurately estimates spectral reflectance and spectral power distribution from single RGB images.
• The approach demonstrates superior performance compared to state-of-the-art learning-based methods on simulated data.
• The method shows improved results on real-world data compared to other interreflection-based techniques and is more robust to noise.

Conclusions:

• Deep learning, when combined with physics-based interreflection modeling, offers a powerful solution for spectral estimation.
• The proposed method provides a robust and accurate way to retrieve surface spectral reflectance and illuminant spectral power distribution.
• This work advances the capabilities of computational imaging for material and light analysis.