InpaintFusion: Incremental RGB-D Inpainting for 3D Scenes.

Area of Science:

• Computer Vision
• Computer Graphics
• Augmented Reality

Background:

• Diminished reality techniques often struggle with non-planar scenes, leading to visual artifacts.
• Existing methods propagate pixel information, which is insufficient for complex geometries.

Purpose of the Study:

• To introduce InpaintFusion, a novel real-time method for inpainting in non-planar scenes.
• To improve diminished reality by incorporating depth information alongside color data.

Main Methods:

• Utilized an RGB-D sensor for simultaneous localization and mapping (SLAM) to capture depth and track camera movement.
• Developed a global optimization approach considering both color and geometric appearance for simultaneous inpainting of color and depth.
• Implemented depth fusion to merge inpainted depth data into a global map.

Main Results:

• InpaintFusion successfully extends inpainting to non-planar scenes, significantly reducing artifacts compared to planar methods.
• The method enables real-time processing for dynamic scene reconstruction.
• The generated map model supports advanced rendering effects like relighting and stereo rendering.

Conclusions:

• InpaintFusion offers a robust solution for real-time inpainting in complex, non-planar environments.
• The integration of depth information is crucial for accurate diminished reality applications.
• The method opens possibilities for enhanced augmented and virtual reality experiences.