Multiplane-Based Cross-View Interaction Mechanism for Robust Light Field Angular Super-Resolution.

Area of Science:

• Computer Vision
• Image Processing
• Computer Graphics

Background:

• Dense sampling of light fields (LF) is crucial for applications like virtual reality but is costly.
• Synthesizing novel views from sparse LF data (LF Angular Super-Resolution or LFASR) is an effective solution.
• Existing LFASR methods struggle with reconstruction quality due to inefficient cross-view interaction.

Purpose of the Study:

• To propose a novel Multiplane-based Cross-view Interaction Mechanism (MCIM) for robust LFASR.
• To improve the accuracy and efficiency of cross-view interaction in LFASR.
• To enhance the network's robustness to scene geometry and suitability for different-baseline LF scenes.

Main Methods:

• Developed a Multiplane Feature Fusion (MPFF) strategy inspired by MultiPlane Images (MPI).
• Implemented a plane selection strategy leveraging MPI transparency to manage information redundancy.
• Designed CSTNet, a Cross-Shaped Transformer-based network utilizing cross-shaped self-attention for efficient LFASR.

Main Results:

• The proposed MCIM method demonstrates superior visual and quantitative performance compared to state-of-the-art methods.
• MPFF facilitates fast and accurate cross-view interaction, improving robustness.
• Experimental results validate state-of-the-art performance on synthetic and real-world LF scenes.

Conclusions:

• The MCIM approach significantly enhances LFASR by enabling efficient cross-view interaction.
• CSTNet, incorporating MCIM, achieves high-quality novel view synthesis from sparse LF data.
• The method is suitable for various LF scenes and offers low-cost training and inference.