Gloss perception: Searching for a deep neural network that behaves like humans

Area of Science:

• Computer Vision
• Human Perception
• Artificial Intelligence

Background:

• Human gloss perception is not fully understood.
• Existing models struggle to replicate human judgments independent of object shape and viewing conditions.
• A computable model is needed to test hypotheses about surface perception.

Purpose of the Study:

• To model human gloss perception using artificial neural networks.
• To identify which computational models best replicate human gloss judgments.
• To understand the visual computations underlying gloss perception.

Main Methods:

• Rendered over 70,000 scenes of objects with varying glossiness.
• Trained various classifiers, including convolutional neural networks (CNNs), to distinguish gloss levels.
• Identified human error patterns and used Bayesian optimization to find CNNs that mimicked these errors.

Main Results:

• Shallower to intermediate depth CNNs (3-5 layers) showed higher correlation with human judgments.
• Deep convolutional generative adversarial networks (DCGANs) with as few as two layers could generate recognizable gloss.
• Human gloss classification aligns with early to mid-level vision computations.

Conclusions:

• Artificial neural networks, particularly CNNs, can effectively model human gloss perception.
• The findings suggest that early to mid-level visual processing is key to gloss discrimination.
• This work provides a framework for further investigation into the mechanisms of surface perception.