Hierarchical abstraction drives human-like 3-D shape processing in deep learning models

Area of Science:

• Computer Vision
• Cognitive Science
• Artificial Intelligence

Background:

• Deep learning models and humans can recognize objects from 3D point clouds.
• Current models achieve human-like performance but may not develop similar 3D shape representations.
• Existing evidence suggests models learn local geometric structures but may have limited global shape understanding.

Purpose of the Study:

• To investigate whether deep learning models develop 3D shape representations comparable to human vision.
• To test the hypothesis that deep learning models have limited representations of 3D global shapes.
• To compare the performance of different deep learning architectures against human performance in 3D shape recognition tasks.

Main Methods:

• Conducted three human experiments manipulating point density, object orientation, local geometry, and part configuration.
• Compared human performance with convolution-based (DGCNN) and transformer-based (Point Transformer) deep learning models.
• Utilized ablation simulations to identify key architectural features driving model performance.

Main Results:

• Human performance remained stable across variations in point density, orientation, and local geometry.
• Human performance significantly declined when object parts were scrambled, indicating sensitivity to global configuration.
• Transformer-based models more accurately replicated human performance patterns across experimental conditions compared to convolution-based models.
• Progressive downsampling in transformer models was identified as crucial for hierarchical shape abstraction.

Conclusions:

• Deep learning models, particularly transformer-based architectures, show promise in developing human-like 3D shape representations.
• Hierarchical abstraction through operations like downsampling is key for models to capture global shape information effectively.
• Further research is needed to fully understand and replicate the nuances of human 3D shape perception in artificial intelligence.