Instant Automatic Emptying of Panoramic Indoor Scenes

Area of Science:

• Computer Vision
• Extended Reality (XR)
• Diminished Reality (DR)

Background:

• 360° cameras are vital for Extended Reality (XR) applications.
• Diminished Reality (DR) techniques are needed to conceal objects in XR.
• Existing methods often focus on single object removal based on semantics.

Purpose of the Study:

• To develop a data-driven approach for automatically removing clutter from 360° indoor scenes.
• To generate photorealistic, clutter-free omnidirectional views with accurate depth maps.
• To achieve low-latency processing for real-time applications.

Main Methods:

• A holistic, data-driven approach applied to the entire 360° scene.
• Calculation of an attention mask based on geometric differences between full and empty scenes.
• Gated convolutions for generating output images and depth, leveraging geometric losses during training.

Main Results:

• The method achieves very low latency and outperforms state-of-the-art solutions in prediction accuracy.
• It successfully separates clutter from architectural structure in a single step.
• Consistent quality results are observed even for real-world scenes without ground truth data.

Conclusions:

• The proposed method offers an efficient and accurate solution for clutter removal in 360° indoor environments.
• It enables advanced Diminished Reality (DR) applications within Extended Reality (XR).
• The lightweight network design ensures interactive performance for practical use.