CSDN: Cross-Modal Shape-Transfer Dual-Refinement Network for Point Cloud Completion.

Area of Science:

• Computer Vision
• 3D Geometry Processing
• Machine Learning

Background:

• Point cloud completion is crucial for reconstructing 3D objects from incomplete data.
• Existing methods often struggle with cross-modal data integration and fine detail generation.

Purpose of the Study:

• To propose a novel coarse-to-fine network for point cloud completion that leverages cross-modal information from images.
• To enhance the accuracy and detail of reconstructed 3D shapes by imitating physical repair processes.

Main Methods:

• Developed the Cross-modal Shape-transfer Dual-refinement Network (CSDN) employing a coarse-to-fine paradigm.
• Integrated a "shape fusion" module using IPAdaIN for cross-modal feature embedding.
• Utilized a "dual-refinement" module with graph convolution and global image constraints for detailed point adjustment.

Main Results:

• CSDN effectively transfers shape characteristics from images to guide point cloud completion.
• The dual-refinement process successfully refines coarse geometry with local and global constraints.
• CSDN demonstrated superior performance compared to twelve competing methods on a cross-modal benchmark.

Conclusions:

• CSDN offers an effective approach for high-quality point cloud completion by exploiting complementary image and point cloud data.
• The proposed method successfully addresses the cross-modal challenge in 3D shape reconstruction.