Light Field Angular Super-Resolution via Spatial-Angular Correlation Extracted by Deformable Convolutional Network.

Area of Science:

• Computer Vision
• Image Processing
• Deep Learning

Background:

• Limited optical sensor resolution restricts simultaneous high spatial and angular resolution in Light Field (LF) images.
• Accurate extraction of Spatial-Angular Correlation (SAC) features is vital for LF image reconstruction in Light Field Angular Super-Resolution (LFASR).
• Existing Deep Neural Network (DNN)-based LFASR methods struggle to extract SAC features effectively from low-angular resolution LF images.

Purpose of the Study:

• To address the limitations of current DNN-based LFASR methods in accurately and completely extracting SAC features.
• To enhance the extraction of SAC features from distant pixels in low-angular resolution LF images.
• To improve the overall quality of reconstructed LF images through advanced feature extraction.

Main Methods:

• Introduced Deformable Convolutional Network (DCN) to adaptively adjust sampling points for capturing SAC features from distant pixels.
• Proposed a Multi-Maximum-Offsets Fusion DCN (MMOF-DCN) to refine offset accuracy and improve SAC feature extraction efficiency.
• Utilized DCN and MMOF-DCN within a DNN framework for LFASR.

Main Results:

• The proposed DCN and MMOF-DCN methods demonstrated superior performance in extracting SAC features compared to existing LFASR techniques.
• Experiments on both real-world and synthetic datasets showed significant advantages of the proposed approach.
• Achieved a 0.45 dB improvement in Peak Signal-to-Noise Ratio (PSNR) on a synthetic dataset with large disparity.

Conclusions:

• The developed MMOF-DCN method effectively addresses the challenges in extracting SAC features for LFASR.
• The proposed approach offers improved accuracy and efficiency in reconstructing high-resolution LF images.
• This work advances the state-of-the-art in LFASR, particularly for images with significant disparity.