Single-View 3D Scene Reconstruction and Parsing by Attribute Grammar

Area of Science:

• Computer Vision
• Computational Geometry
• Artificial Intelligence

Background:

• Image understanding and 3D scene reconstruction are complex tasks.
• Existing methods often address these tasks separately, limiting performance.
• A unified approach is needed for robust scene analysis.

Purpose of the Study:

• To develop an attribute grammar for joint 2D image parsing and 3D scene structure recovery.
• To represent 3D spatial relationships between planar surfaces within a hierarchical parse graph.
• To enable simultaneous optimization of image recognition and 3D reconstruction.

Main Methods:

• An attribute grammar with production rules defining spatial relations between 3D planar surfaces.
• Augmenting parse graph nodes with attribute variables for global and local scene geometry.
• Utilizing a probabilistic framework and Markov Chain Monte Carlo (MCMC) for parse graph construction.

Main Results:

• The proposed grammar successfully decomposes images into hierarchical parse graphs.
• Attribute variables effectively encode geometric constraints within the scene structure.
• The MCMC method optimizes for both 2D recognition and 3D reconstruction concurrently.

Conclusions:

• The attribute grammar provides a powerful framework for coupled image parsing and 3D scene reconstruction.
• The method achieves state-of-the-art results on benchmark and new datasets.
• This approach offers a unified solution for single-image 3D scene understanding.