Joint 3D shape and hyperspectral reconstruction from single-view spectro-polarimetric imaging

Shape from Polarization (SfP) has emerged as a promising technique for reconstructing the three-dimensional (3D) structure of objects using polarization information. However, existing 3D reconstruction methods generally suffer from degraded accuracy when dealing with dark or low-reflectivity surfaces, resulting in unreliable point cloud data. Furthermore, most current methods generally focus on geometric reconstruction, such as surface normals or depth maps, limiting their ability to capture the spectral characteristics of objects' materials. Here, we propose a novel single-view spectro-polarimetric imaging (SVSPI) method that, for the first time to our knowledge, achieves joint reconstruction of 3D shape and hyperspectral information from single-view spectro-polarimetric imaging for dark-surfaced objects. This method uses an on-chip broadband hyperspectral imaging sensor and introduces a dedicated spectral-polarization-3D dataset to support training and evaluation. Experimental results demonstrate the effectiveness of the proposed method, achieving the simultaneous reconstruction of surface normals, depth maps, and spectral information within the 400-1000 nm range under complex environmental conditions.