Shape from polarization based on a polarization representation and sparse self-attention.

Area of Science:

• Computer Vision
• Photogrammetry
• 3D Reconstruction

Background:

• Physics-based shape from polarization (SfP) methods struggle with mixed reflections and local ambiguities.
• Deep learning-based SfP methods offer improved accuracy but lack global context and physical prior integration.
• Accurate surface normal estimation is vital for 3D reconstruction.

Purpose of the Study:

• To develop a novel learning-based SfP method enhancing shape recovery accuracy.
• To combine physical priors with sparse self-attention for improved SfP performance.
• To address limitations in global context perception and physical prior utilization in learning-based SfP.

Main Methods:

• Introduced a novel polarization representation using Stokes vectors for efficient physical prior utilization.
• Incorporated a sparse self-attention mechanism with bi-level routing to capture global context and resolve ambiguities.
• Employed spatial and channel attention mechanisms for optimized feature fusion and high-frequency detail capture.

Main Results:

• The proposed method outperforms state-of-the-art SfP methods on the DeepSfP dataset and a self-built dataset.
• Achieved a mean angular error of 12.06° on the DeepSfP dataset.
• Demonstrated significant enhancement in normal estimation accuracy.

Conclusions:

• The novel SfP method effectively integrates physical priors and sparse self-attention for superior shape recovery.
• The proposed techniques significantly improve normal estimation accuracy, offering robust technical support for SfP tasks.
• This approach advances the field of 3D reconstruction using polarization information.