CrossAlignNet: a self-supervised feature learning framework for 3D point cloud understanding.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Data Analysis

Background:

• Existing methods struggle with imbalanced learning of global semantic and local geometric features in point clouds.
• Cross-modal information asymmetry hinders effective point cloud understanding when integrating with other data types.

Purpose of the Study:

• To propose CrossAlignNet, a self-supervised framework for point cloud representation learning.
• To address the limitations of existing methods in balancing global and local feature learning and cross-modal information.
• To enhance point cloud understanding through a novel cross-modal mask alignment strategy.

Main Methods:

• Developed a synchronized mask alignment strategy to create geometrically consistent regions between point cloud and image patches.
• Implemented a dual-task learning framework: global semantic alignment via contrastive learning and local mask reconstruction using cross-attention.
• Introduced the ShapeNet3D-CMA dataset with precise point cloud-image spatial mappings for cross-modal learning.

Main Results:

• CrossAlignNet demonstrates superior or comparative performance on object classification, few-shot classification, and part segmentation tasks.
• The framework effectively achieves cross-modal information symmetry and balances feature learning.
• The proposed ShapeNet3D-CMA dataset facilitates robust cross-modal learning.

Conclusions:

• CrossAlignNet offers an effective self-supervised approach for point cloud representation learning.
• The cross-modal mask alignment strategy significantly improves the integration of geometric and semantic information.
• The framework advances the state-of-the-art in various 3D point cloud understanding applications.