Joint 3D shape and hyperspectral reconstruction from single-view spectro-polarimetric imaging

Area of Science:

• Computer Vision
• Optical Engineering
• Material Science

Background:

• Shape from Polarization (SfP) is a 3D reconstruction technique.
• Current methods struggle with dark or low-reflectivity surfaces.
• Existing methods primarily focus on geometry, neglecting spectral properties.

Purpose of the Study:

• To develop a novel method for joint 3D shape and hyperspectral reconstruction.
• To address limitations of SfP for dark-surfaced objects.
• To enable simultaneous capture of geometric and spectral information.

Main Methods:

• Proposed a single-view spectro-polarimetric imaging (SVSPI) method.
• Utilized an on-chip broadband hyperspectral imaging sensor.
• Developed a dedicated spectral-polarization-3D dataset for training and evaluation.

Main Results:

• Achieved joint reconstruction of 3D shape (surface normals, depth maps) and hyperspectral information (400-1000 nm).
• Demonstrated effectiveness under complex environmental conditions.
• Successfully reconstructed data for dark-surfaced objects.

Conclusions:

• The SVSPI method offers a significant advancement in 3D reconstruction for challenging surfaces.
• This technique enables comprehensive material characterization by combining shape and spectral data.
• The developed dataset supports further research in spectro-polarimetric imaging.