Variation in the geometry of concept manifolds across human visual cortex

Area of Science:

• Neuroscience
• Computer Vision
• Machine Learning

Background:

• Accurate linear classification of visual concepts in the brain depends on the geometry of concept manifolds.
• Understanding how these geometric properties differ between brain regions and deep neural networks (DNNs) is crucial.

Purpose of the Study:

• To investigate the geometric properties of concept manifolds in human visual cortex and DNNs.
• To determine how these properties influence few-shot linear classification accuracy.
• To compare the geometric strategies employed by the brain and DNNs.

Main Methods:

• Utilized a large fMRI dataset to estimate geometric properties of concept manifolds in the human visual cortex.
• Analyzed geometric properties of concept manifolds in deep neural networks (DNNs).
• Applied a recent theory linking manifold geometry to few-shot linear classification accuracy.

Main Results:

• Classification accuracy variation in human visual cortex is driven by the distance between manifold centers (geometric Signal).
• Classification accuracy variation in DNN layers is primarily driven by the effective number of manifold dimensions (Dimensionality).
• A strong negative correlation exists between Signal and Dimensionality across both brain regions and DNN layers.

Conclusions:

• Human visual cortex and DNNs employ distinct geometric strategies for linear classification of visual concepts.
• Despite differing strategies, both systems operate under a shared constraint linking Signal and Dimensionality.
• This research provides insights into the computational principles underlying visual processing in biological and artificial systems.