Robust Shape Fitting for 3D Scene Abstraction.

Area of Science:

• Computer Vision
• 3D Scene Understanding
• Geometric Modeling

Background:

• Humans perceive environments as arrangements of simple parametric models, like cuboids.
• Inferring these primitives is key for abstract scene descriptions.
• Existing methods struggle with complex scenes and simple object reproduction.

Purpose of the Study:

• To develop a robust primitive fitting estimator for abstracting complex real-world environments using cuboids.
• To improve upon existing methods that directly estimate shape parameters and fail with complex objects.

Main Methods:

• A RANSAC estimator guided by a neural network fits cuboid primitives to depth maps.
• The network is conditioned on previously detected scene parts for sequential parsing.
• An end-to-end optimized depth estimation Convolutional Neural Network (CNN) is used for RGB images.
• An improved occlusion-aware distance metric handles opaque scenes effectively.
• A neural network-based cuboid solver enhances parsimony and reduces inference time.

Main Results:

• The algorithm successfully abstracts cluttered real-world 3D scene layouts.
• The proposed method provides more parsimonious scene abstractions.
• Inference time is reduced compared to previous approaches.
• The method does not require labor-intensive cuboid annotations for training.

Conclusions:

• The developed robust estimator meaningfully abstracts complex 3D scenes using cuboids.
• The occlusion-aware metric and neural network solver significantly improve primitive fitting.
• This approach offers a more efficient and accurate method for 3D scene abstraction.