Metameric Inpainting for Image Warping

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Perceptual Computing

Background:

• Image warping is crucial for immersive experiences like VR/AR.
• Disocclusions (unknown image areas) pose a significant challenge in image warping.
• Existing inpainting methods lack real-time performance or visual fidelity.

Purpose of the Study:

• To develop a real-time image warping method addressing disocclusions.
• To leverage human visual perception for realistic hole-filling.
• To improve upon existing structured and color-based inpainting techniques.

Main Methods:

• Introduced Metameric Image Inpainting, a novel approach for real-time hole-filling.
• Estimated image feature statistics from neighboring regions for disoccluded areas.
• Inpainted these statistics to synthesize visually plausible content.

Main Results:

• Achieved real-time performance for image warping with disocclusion handling.
• Synthesized visuals were less noticeable to participants, especially in peripheral vision.
• Demonstrated improved realism compared to color-based methods and speed over structured methods.

Conclusions:

• Metameric Image Inpainting offers a viable solution for real-time disocclusion handling in image warping.
• The method's foundation in human perception enhances visual realism.
• Enables advancements in applications like 6-DoF rendering and remote render-streaming.