Hyperpixels: Flexible 4D Over-Segmentation for Dense and Sparse Light Fields.

Area of Science:

• Computer Vision
• Image Processing
• Computational Photography

Background:

• 4D Light Field (LF) imaging captures spatial and angular information, crucial for advanced computer vision and immersive experiences.
• Existing methods for representing LF data often struggle with sparse sampling and occlusions, and do not fully leverage spatio-angular cues.
• Image over-segmentation has shown promise for LF representation, but current techniques are limited by assumptions of dense sampling.

Purpose of the Study:

• To propose a flexible, automatic, and adaptive representation for 4D Light Field data, suitable for both dense and sparse scenarios.
• To address the limitations of existing methods in handling sparse LF data and occlusions.
• To fully exploit spatio-angular LF cues for improved representation.

Main Methods:

• Introduction of the 'hyperpixel' concept for LF data representation.
• Estimation of disparity maps for all views to improve over-segmentation accuracy and consistency.
• Application of a modified weighted K-means clustering in 4D Euclidean space using robust spatio-angular features.

Main Results:

• The proposed hyperpixel representation demonstrates flexibility and adaptivity for both dense and sparse 4D LFs.
• Experimental results show competitive and superior performance compared to state-of-the-art methods.
• Achieved improvements in over-segmentation accuracy, shape regularity, and view consistency.

Conclusions:

• The hyperpixel method offers a robust and effective way to represent 4D Light Field data, overcoming limitations of previous approaches.
• This new representation facilitates subsequent computer vision applications by providing a more comprehensive spatio-angular data structure.
• The approach is validated on diverse datasets, showcasing its effectiveness for sparse and occluded LF imaging.