MCTS With Refinement for Proposals Selection Games in Scene Understanding.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Computational Geometry

Background:

• Scene understanding tasks like floor plan reconstruction are complex.
• Existing methods often rely on strong assumptions or constraints.
• Monte Carlo Tree Search (MCTS) is effective for high-state complexity games but underexplored in scene understanding.

Purpose of the Study:

• To adapt Monte Carlo Tree Search (MCTS) for general scene understanding problems.
• To develop a novel method for joint selection and optimization of proposals.
• To improve the accuracy and efficiency of 2D floor plan reconstruction and 3D room layout generation.

Main Methods:

• Adapted Monte Carlo Tree Search (MCTS) for joint proposal selection and optimization.
• Developed a novel differentiable renderer for 2D polygonal room proposals.
• Applied the method to floor plan reconstruction from point clouds and 3D room layout generation from images.

Main Results:

• Achieved significant improvements in speed and quality for floor plan reconstruction on Structured3D and Floor-SP datasets.
• Demonstrated accurate 3D room layout reconstruction from color images on the Matterport3D-Layout dataset.
• Showcased the adaptability of the differentiable renderer for 2D and 3D planar polygons.

Conclusions:

• The proposed MCTS-based method effectively addresses scene understanding challenges.
• The approach offers state-of-the-art performance without imposing restrictive constraints on layouts.
• The differentiable renderer is a versatile tool for geometric shape rendering.