Efficient Reflectance Capture With a Deep Gated Mixture-of-Experts

Area of Science:

• Computer Vision
• Computer Graphics
• Material Science

Background:

• Accurate material property acquisition is crucial for realistic rendering and analysis.
• Existing methods often require numerous input images or lack adaptability to diverse materials.
• Unified deep learning models may sacrifice quality for generality.

Purpose of the Study:

• To develop an efficient framework for pixel-independent anisotropic reflectance acquisition.
• To enhance reconstruction quality by conditioning network behavior on input photometric measurements.
• To reduce the number of required input images for high-quality reflectance reconstruction.

Main Methods:

• A deep gated mixture-of-experts framework was designed.
• A gating module selects specialized decoders based on photometric measurements.
• Illumination conditions during acquisition were jointly optimized.
• The framework was extended to handle non-planar reflectance scanning.

Main Results:

• The proposed method achieves improved quality compared to state-of-the-art techniques using the same number of input images.
• The number of input images can be reduced by approximately two-thirds for comparable results.
• The framework demonstrates effectiveness on various challenging near-planar samples using a lightstage.
• Generalization to enhance non-planar reflectance scanning was shown.

Conclusions:

• The novel gated mixture-of-experts approach enables efficient and high-quality anisotropic reflectance acquisition.
• The input-adaptive nature of the network leads to superior reconstruction performance.
• This framework offers a significant improvement in terms of quality and data efficiency for reflectance scanning.