End-to-end deep unfolding network for DoFP polarization image reconstruction

Area of Science:

• Optics and Photonics
• Computer Vision
• Machine Learning

Background:

• Snapshot polarimetric imaging systems, particularly those using division of focal plane (DoFP) sensors, capture polarization information but suffer from reduced spatial resolution due to micro-polarizer arrays.
• Existing learning-based methods for polarization image reconstruction often overlook the underlying physical principles governing the process.

Purpose of the Study:

• To develop a novel deep unfolding network, DUPIR, for end-to-end polarization image reconstruction.
• To jointly reconstruct high-resolution intensity images (I0, I45, I90, I135) and polarization parameters (Stokes S0, DoLP, AoP).
• To integrate physical model priors into a trainable deep learning architecture.

Main Methods:

• A deep unfolding network (DUPIR) was proposed, combining model-based priors with a learning-based approach.
• The network was trained end-to-end to reconstruct four linear polarization orientations and key polarization parameters.
• A new polarization dataset with 184 sample pairs was created to address data scarcity.

Main Results:

• DUPIR demonstrated state-of-the-art reconstruction accuracy on both public and newly collected datasets.
• The method successfully achieved real-time inference capabilities.
• The integrated physical priors improved the reconstruction quality compared to methods neglecting them.

Conclusions:

• DUPIR effectively bridges the gap between model-based and learning-based methods for polarization image reconstruction.
• The proposed network offers a robust solution for enhancing spatial resolution in DoFP systems.
• The developed dataset and method contribute to advancing polarization imaging research.