Seamless Texture Optimization for RGB-D Reconstruction

Restoring high-fidelity textures for 3D reconstructed models are an increasing demand in AR/VR, cultural heritage protection, entertainment, and other relevant fields. Due to geometric errors and camera pose drifting, existing texture mapping algorithms are either plagued by blurring and ghosting or suffer from undesirable visual seams. In this paper, we propose a novel tri-directional similarity texture synthesis method to eliminate the texture inconsistency in RGB-D 3D reconstruction and generate visually realistic texture mapping results. In addition to RGB color information, we incorporate a novel color image texture detail layer serving as an additional context to improve the effectiveness and robustness of the proposed method. First, we select an optimal texture image for each triangle face of the reconstructed model to avoid texture blurring and ghosting. During the selection procedure, the texture details are weighted to avoid generating texture chart partitions across high-frequency areas. Then, we optimize the camera pose of each texture image to align with the reconstructed 3D shape. Next, we propose a tri-directional similarity function to resynthesize the image context within the boundary stripe of texture charts, which can significantly diminish the occurrence of texture seams. Finally, we introduce a global color harmonization method to address the color inconsistency between texture images captured from different viewpoints. The experimental results demonstrate that the proposed method outperforms state-of-the-art texture mapping methods and effectively overcomes texture tearing, blurring, and ghosting artifacts.