Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons.

Area of Science:

• Neuroscience
• Computer Vision
• Machine Learning

Background:

• Understanding face perception in the brain is crucial for neuroscience.
• The ventral visual stream's learning objectives remain unclear.
• Neural responses to faces in the inferotemporal cortex are complex.

Purpose of the Study:

• To investigate the learning objectives driving neural representations of faces in the ventral visual stream.
• To model neural responses to faces using a deep generative model.
• To compare model-generated factors with neural coding in the inferotemporal cortex.

Main Methods:

• Utilized a deep self-supervised generative model, beta-Variational Autoencoder (β-VAE), to disentangle sensory data into latent factors.
• Modeled neural responses to faces in the macaque inferotemporal (IT) cortex.
• Compared β-VAE's discovered factors with neural coding of single IT neurons and baseline models (Active Appearance Model, deep classifiers).

Main Results:

• Demonstrated a strong correspondence between β-VAE's disentangled generative factors (e.g., gender, age) and those coded by single IT neurons.
• Achieved superior performance compared to baseline models in explaining neural responses.
• Showcased β-VAE's ability to reconstruct novel face images from limited neural signals.

Conclusions:

• Optimizing a disentangling objective yields representations that closely mirror those found in the IT cortex at the single-unit level.
• Disentangling is proposed as a plausible learning objective for the visual brain in face perception.
• This work provides insights into the computational principles underlying biological vision.