Zero-Shot 3D Object Classification via Graph-Based Local Geometric Features and Depth-Aware Multi-View Projection

Area of Science:

• Computer Vision
• Machine Learning
• 3D Data Analysis

Background:

• 3D point cloud classification faces challenges due to limited labeled data and high computational costs.
• Leveraging 2D pre-trained models via projection is a promising strategy for zero-shot 3D classification.
• Generating high-fidelity 2D projections from 3D point clouds remains a critical hurdle.

Purpose of the Study:

• To propose an effective zero-shot 3D object classification framework.
• To address the challenge of generating high-fidelity 2D projections from 3D point clouds.
• To improve the performance of 3D point cloud classification with limited data.

Main Methods:

• A novel framework combining multi-scale local geometric feature extraction and depth-aware multi-view projection.
• Modeling 3D point clouds as graph structures with multi-scale radii for feature extraction.
• Generating multi-view 2D images via depth-wise feature accumulation and using a frozen CLIP model for classification.

Main Results:

• The proposed method effectively preserves structural characteristics of point clouds and reduces the domain gap between 3D and 2D data.
• Experimental results on ModelNet10, ModelNet40, and ScanObjectNN datasets demonstrate superior performance compared to existing methods.
• The framework achieves state-of-the-art results in zero-shot 3D point cloud classification.

Conclusions:

• The proposed framework offers an effective solution for zero-shot 3D object classification, particularly in complex scenarios.
• Depth-aware multi-view projection and multi-scale feature extraction are key to high-fidelity 2D representations.
• This approach significantly advances the capabilities of point cloud analysis with limited labeled data.