Human gloss perception reproduced by tiny neural networks

Area of Science:

• Visual neuroscience
• Machine learning
• Cognitive science

Background:

• Understanding how the brain infers object properties like gloss is a key goal in visual neuroscience.
• Human gloss judgments are complex and challenging to explain computationally.
• Previous models often focused on replicating physical reality rather than human perception.

Purpose of the Study:

• To identify the computations underlying human gloss judgments using machine learning.
• To compare the effectiveness of different neural network architectures in predicting human gloss perception.
• To investigate whether simple or complex computations drive gloss perception.

Main Methods:

• Generated thousands of 3D object renderings with varying shapes, lighting, and viewpoints using a Ward reflectance model.
• Collected human gloss ratings for each rendered image.
• Trained two types of neural networks: 'ground-truth networks' to estimate physical reflectance and 'human-like networks' to reproduce human judgments.

Main Results:

• Human gloss judgments were consistent but systematically deviated from physical reality.
• Shallow neural networks accurately replicated human gloss judgments.
• A single-filter network outperformed deep 'ground-truth' networks in predicting human judgments and generalized to gloss illusions.

Conclusions:

• Gloss perception appears to rely on simple, general-purpose computations rather than complex ones.
• Interpretable, 'tiny' neural networks are powerful tools for understanding cognitive processes like gloss perception.
• This study offers new insights into the mechanisms of visual inference in the human brain.