Diving Into Epipolar Transformers for Light Field Super-Resolution and Disparity Estimation.

Area of Science:

• Computer Vision
• Image Processing
• Geometric Deep Learning

Background:

• Light field (LF) cameras capture multi-view 3D scene data, offering enhanced immersion over traditional cameras.
• Modeling non-local spatial-angular correlations in LF images is challenging, especially with complex disparity variations.

Purpose of the Study:

• To propose a novel Epipolar Transformer mechanism for LF image processing.
• To effectively model geometrically meaningful correlations along epipolar lines in LF images.
• To improve performance in LF super-resolution and disparity estimation tasks.

Main Methods:

• Developed a generic Epipolar Transformer mechanism leveraging orthogonal epipolar geometry.
• Incorporated geometrically meaningful correlations along epipolar lines.
• Applied the transformer to LF spatial/angular super-resolution and disparity estimation.

Main Results:

• Achieved state-of-the-art performance on benchmark datasets for LF super-resolution.
• Demonstrated robust performance on large disparity variations for LF super-resolution.
• Enabled direct disparity regression, overcoming limitations of fixed maximum disparity in estimation.

Conclusions:

• The Epipolar Transformer learns effective LF feature representations without redundant designs.
• The mechanism is flexible and adaptable to various LF-related tasks.
• The proposed method significantly advances LF image processing capabilities.