PlenoPatch: Patch-Based Plenoptic Image Manipulation

Area of Science:

• Computer Vision
• Computer Graphics
• Image Processing

Background:

• Patch-based image synthesis is effective for editing 2D images, videos, and stereo pairs.
• Light field editing traditionally requires specialized tools and expertise.
• Consumer-level lenselet devices capture plenoptic images, offering rich 3D information.

Purpose of the Study:

• To extend patch-based synthesis for interactive and efficient light field editing.
• To enable novice users to edit plenoptic images using familiar 2D interfaces.
• To achieve realistic editing results, including handling occlusions and semi-transparency.

Main Methods:

• Representing light fields as a set of viewpoint images.
• Decomposing the central view into depth layers for user interaction.
• Applying patch-based synthesis to affected layers and propagating edits across views.
• Handling object boundary occlusions with semi-transparency.

Main Results:

• Successful demonstration of interactive light field editing on consumer-level devices.
• Realistic results for applications like hole-filling, object manipulation, and depth adjustment.
• Effective propagation of edits across all viewpoints with occlusion handling.

Conclusions:

• Patch-based synthesis is a viable and efficient method for light field editing.
• The proposed approach democratizes light field manipulation for a wider user base.
• This technique significantly enhances realism and usability in plenoptic image editing.