Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Data Analysis

Background:

• Few-shot and zero-shot learning in 3D point cloud semantic segmentation face challenges due to limited annotated 3D datasets.
• Existing 2D methods are less effective in 3D due to less representative features and high intra-class variation.
• The cost of 3D data collection and annotation hinders the development of robust 3D deep learning models.

Purpose of the Study:

• To develop effective methods for few-shot and zero-shot 3D point cloud semantic segmentation.
• To address the limitations of feature representation and intra-class variation in 3D point cloud data.
• To improve the performance of semantic segmentation on sparse and limited 3D datasets.

Main Methods:

• Proposed a Query-Guided Prototype Adaption (QGPA) module to adapt prototypes between support and query point cloud feature spaces.
• Introduced a Self-Reconstruction (SR) module to enhance prototype representation by reconstructing support masks.
• Developed a semantic-visual projection model for zero-shot segmentation by incorporating category words as semantic information.

Main Results:

• The QGPA module significantly alleviates large feature intra-class variation in point clouds.
• The proposed method achieved substantial performance gains, surpassing state-of-the-art algorithms.
• Achieved 7.90% and 14.82% improvements on S3DIS and ScanNet benchmarks under the 2-way 1-shot setting, respectively.

Conclusions:

• The proposed QGPA and SR modules effectively enhance few-shot 3D point cloud semantic segmentation.
• The semantic-visual projection model enables effective zero-shot segmentation by bridging semantic and visual information.
• The developed approach offers a significant advancement in handling limited data scenarios for 3D semantic segmentation.