Learning Shape Anchors for Holistic Indoor Scene Understanding

This paper presents a shapeanchor guided learning strategy (AncLearn) for robust holistic indoor scene understanding. We observe that the search space constructed by current methods for proposal feature grouping and instance point sampling often introduces massive noise to instance parsing and object reconstruction.Accordingly, we develop AncLearn to learn the shape anchors of 3D objects, to provide shape constraints in a top-down manner. The learned anchors fit instance surfaces and thus provide fine localization clues for (i) separating noise and object-related features to offer reliable instance proposals, (ii) reducing outliers in object point sampling to provide well-structured geometry priors for object reconstruction and (iii) integrating RGB information at the instance level to incorporate additional texture clues for better semantic perception.We embed the AncLearn strategy into a reconstruction-from-detection learning system (AncRec++) to generate high-quality semantic scene models in a purely instance-oriented manner. Within AncLearn, the network learns to abstract the shape priors of each instance and thus improves its robustness against the sparsity and incompleteness of point clouds. Experiments conducted on the challenging ScanNetv2 dataset demonstrate that our method consistently achieves state-of-the-art performance in terms of 3D object detection, layout estimation, and shape reconstruction.