Learning to remove occlusions in light field images using multiscale receptive fields and feature pyramid networks

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Occlusion removal in light-field (LF) images is crucial for scene understanding.
• Existing methods are limited by small receptive fields, hindering long-range dependency capture.
• Effective reconstruction of occluded regions remains a challenge.

Purpose of the Study:

• To propose LF-PyrNet, an end-to-end deep learning model for enhanced occlusion removal in LF images.
• To improve the ability to capture long-range dependencies and reconstruct occluded areas.
• To leverage multi-scale receptive field learning and hierarchical feature refinement.

Main Methods:

• Feature extraction using Residual Atrous Spatial Pyramid Pooling (ResASPP) and receptive field blocks (RFB) to expand receptive fields.
• Occlusion reconstruction via cascaded Residual Dense Blocks (RDBs) with densely connected layers.
• Multi-scale feature fusion and refinement using a Feature Pyramid Network (FPN) and a refinement module with separable/standard convolutions.

Main Results:

• Expanded receptive fields significantly improve occlusion removal performance.
• LF-PyrNet demonstrates effective capture of broader context and multi-scale spatial dependencies.
• Enhanced structural consistency and texture restoration in occluded regions are achieved.

Conclusions:

• LF-PyrNet offers a reliable solution for reconstructing occluded regions in LF images.
• The proposed architecture effectively addresses limitations of restricted receptive fields.
• Multi-scale receptive field learning is key to advancing LF image occlusion removal.