Multispectral polarization image demosaicing using redundant Stokes representation

This paper proposes a deep-learning-based demosaicing algorithm, multispectral polarization demosaicing with redundant Stokes (MPD-RS), designed for multispectral polarization filter arrays. The proposed MPD-RS effectively learns the correlation across spatial, spectral, and polarization domains, utilizing a newly constructed dataset of multispectral polarization images (MSPIs). Initially, MPD-RS performs interpolation using a position-variant convolutional kernel to generate a preliminary MSPI. This is followed by conversion to a new Stokes representation, to our knowledge, where the data is decomposed into four components, including a term to capture polarization redundancy. The intensity component is processed with a multi-stage three-dimensional convolutional network, while the remaining components are handled by a lightweight, attention-based network. Experimental results validate the effectiveness of MPD-RS, demonstrating superior peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) for MSPI reconstruction, with an average PSNR improvement of 3.873 dB over the Global Cross-Attention Network, as well as reduced mean squared error in Stokes parameters. The method maintains high accuracy across images with a diverse range of polarization levels, highlighting its adaptability.