Learning Dense Correspondences for Non-Rigid Point Clouds With Two-Stage Regression

We propose a novel deep learning method to predict dense correspondences for partial point clouds of non-rigidly deformable targets. Dense correspondences are learned in the form of vertex displacements of a template mesh towards the point clouds. A two-stage regression framework is proposed to estimate accurate displacement vectors, including the global and local regression networks. Specifically, the global regression network estimates global displacements from the global features of the template mesh and point clouds through a graph CNN based hierarchical encoder-decoder network. Based on the initial displacements, a mesh can be generated that fits to the point clouds roughly. In the local regression network, a local feature embedding layer fuses local features of point clouds with graph features on the generated mesh through an attention mechanism. Consequently, the embedded local features are employed to refine the correspondences in local regions of the targets by predicting the increments of vertex displacements. Our method is further generalized to correspondence estimation on unseen real data with a robust fine-tuning method. The experimental results on diverse datasets of various deformable subjects (e.g., human bodies, animals, and hands) demonstrate that the proposed approach can accurately and robustly estimate dense correspondences from non-rigid point clouds.