Combining Gaussian and Pixel Representation for Light Field View Reconstruction.

Area of Science:

• Computer Vision
• Computer Graphics
• Image Processing

Background:

• Light field (LF) imaging offers rich spatial and angular data, crucial for advanced applications.
• Current LF view reconstruction methods often overlook scene geometry, limiting resolution and accuracy.
• Existing techniques primarily focus on pixel-level modeling, neglecting geometric nuances.

Purpose of the Study:

• To develop a generalizable LF view reconstruction method that effectively models scene geometry.
• To improve imaging resolution in LF applications by integrating 3D Gaussian Splatting principles.
• To achieve both geometric structure and texture detail consistency in reconstructed LF views.

Main Methods:

• Introduced LFGaussian, a network leveraging 3D Gaussian Splatting for LF view reconstruction.
• Proposed disparity-guided 2D Gaussian propagation with a novel Gaussian primitive definition for camera parameter-free operations.
• Implemented a dual-branch workflow: Gaussian representation rendering and pixel representation upsampling for complementary feature extraction.
• Designed sub-sampling Gaussian decoding and Gaussian splitting for efficient and high-quality Gaussian rendering, especially in complex regions.

Main Results:

• LFGaussian demonstrated superior performance over state-of-the-art methods on both real-world and synthetic LF datasets.
• The method effectively achieves geometric structure consistency and texture detail consistency across target views.
• Achieved arbitrary-scale reconstruction, showcasing significant flexibility and high performance.

Conclusions:

• Integrating Gaussian representation, inspired by 3D Gaussian Splatting, is highly effective for LF view reconstruction.
• LFGaussian provides a robust and flexible solution for enhancing LF imaging resolution and quality.
• The proposed approach overcomes limitations of pixel-centric methods by emphasizing geometry modeling.