Inference-Reconstruction Variational Autoencoder for Light Field Image Reconstruction.

Area of Science:

• Computer Vision
• Image Processing
• Computational Photography

Background:

• Light field cameras capture light ray information, enabling advanced 3D perception.
• Existing methods struggle to achieve high spatial and angular resolution simultaneously due to sensor limitations.

Purpose of the Study:

• To develop a novel method for reconstructing dense light field images from limited reference views.
• To overcome the spatial-angular resolution trade-off in current light field imaging.

Main Methods:

• Proposed an inference-reconstruction variational autoencoder (IR-VAE) with interconnected inference and reconstruction networks.
• Introduced Mean Local Maximum Mean Discrepancy (MLMMD) for comparing high-resolution latent variable distributions.
• Developed a viewpoint-dependent indirect view synthesis method using adaptive convolution.

Main Results:

• The IR-VAE effectively reconstructs dense light field images from four corner reference views.
• MLMMD accurately measures statistical distances in high-dimensional latent spaces.
• The indirect view synthesis method achieves efficient and high-quality novel view generation.
• Experimental results demonstrate superior performance over state-of-the-art methods on various light field datasets.

Conclusions:

• The proposed IR-VAE framework significantly advances light field image reconstruction.
• The novel statistical measurement and synthesis methods contribute to improved light field processing.
• This work offers a promising direction for high-resolution light field imaging and 3D perception.