Learning Photometric Feature Transform for Free-Form Object Scan.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Photometric Stereo

Background:

• Multi-view stereo (MVS) pipelines often struggle with varying illumination and unstructured data.
• Extracting robust, view-invariant features is crucial for accurate 3D shape and reflectance reconstruction.

Purpose of the Study:

• To develop a novel framework for automatically learning to aggregate and transform photometric measurements.
• To enhance 3D reconstruction quality by generating spatially distinctive, view-invariant low-level features.
• To reconstruct both geometry and anisotropic reflectance from challenging, hand-held scans.

Main Methods:

• A novel framework jointly trains illumination conditions and feature transforms on synthetic data.
• Features are fed into a multi-view stereo pipeline for improved 3D reconstruction.
• A system is developed to reconstruct geometry and anisotropic reflectance from hand-held scans.

Main Results:

• The system demonstrates effectiveness using a lightweight prototype (camera, LEDs, tablet).
• Reconstructions are validated against professional 3D scanners and photographs.
• The proposed method favorably compares with state-of-the-art techniques in 3D reconstruction.

Conclusions:

• The framework successfully generates view-invariant features for enhanced 3D reconstruction.
• The system enables robust reconstruction of geometry and reflectance from challenging datasets.
• This approach offers a significant improvement over existing multi-view stereo techniques.