Full-resolution image restoration for light field images via a spatial shift-variant degradation network

Area of Science:

• Computational Imaging
• Optics and Photonics
• Computer Vision

Background:

• Light field (LF) imaging systems inherently trade off spatial and angular resolution due to sensor limitations.
• Existing methods for enhancing sub-aperture image (SAI) resolution often neglect the spatial shift-variant nature of LF data.
• Restoring a full-resolution (FR) image from LF data remains a significant challenge.

Purpose of the Study:

• To propose a novel FR image restoration method for LF imaging systems.
• To address the limitations of current methods by considering spatial shift-variant characteristics.
• To develop a network that effectively recovers high-resolution images from degraded LF data.

Main Methods:

• Derived an explicit convolution model using scalar diffraction theory to compute system response and imaging matrices.
• Established a mapping from FR images to SAIs using spatial shift-variant degradation (SSVD) kernels.
• Embedded SSVD kernels into a neural network, featuring a specialized SSVD convolution layer and a refinement block.

Main Results:

• The proposed network demonstrates superior performance in restoring FR images compared to existing methods on simulated and real-world LF data.
• The SSVD convolution layer effectively handles view-wise degradation and accelerates training.
• The refinement block successfully preserves intricate image details.

Conclusions:

• The developed method effectively restores high-quality FR images from LF data by incorporating SSVD kernels.
• The network's ability to handle shift-variant degradation offers a significant advancement in LF image restoration.
• The approach is robust and applicable to various imaging conditions, including multi-focus scenes.