Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Light fields capture 3D scene information, offering immersive perception but generating large data volumes.
• Efficient compression methods are crucial for practical light field applications.

Purpose of the Study:

• To design a compact neural network representation for effective light field compression.
• To achieve high-quality reconstruction of light fields while minimizing data size.

Main Methods:

• A novel neural network architecture using descriptive and modulatory kernels.
• Techniques including modulator allocation, kernel tensor decomposition, non-uniform quantization, and entropy coding.
• Supervised training using randomly initialized noise as input to reconstruct target Sub-Aperture Images (SAIs).

Main Results:

• The proposed method significantly outperforms state-of-the-art (SOTA) methods in light field compression.
• Achieved high-quality decoded light fields with enhanced compactness.
• Demonstrated successful transfer learning of learned modulators for view synthesis.

Conclusions:

• The compact neural network representation is highly effective for light field compression.
• The method offers a promising solution for efficient 3D scene data handling and view synthesis.
• Potential for adapting learned components to new light fields for novel view generation.