Human and artificial visual systems share a computational principle for transforming binocular disparity into depth...

Area of Science:

• Neuroscience
• Computational Vision
• Human Visual Cortex

Background:

• The visual brain deciphers binocular disparity to perceive depth.
• Early visual cortex neurons (V1) use cross-correlation, responding to both matched and mismatched image features, creating an ambiguous depth representation.
• This representation is refined through nonlinear cross-matching computations in later visual areas to filter mismatches.

Purpose of the Study:

• To investigate the organization of depth representations in the human visual cortex.
• To compare the computational principles of human stereo vision with those of deep neural networks (DNNs).

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to map brain activity.
• A deep neural network (DNN) trained for stereo vision was analyzed for comparison.

Main Results:

• Areas V1-V3 showed stronger cross-correlation components, characteristic of initial disparity processing.
• Areas V3A/B, V7, hV4, and hMT+ demonstrated a shift towards cross-matching computations, indicating refined depth processing.
• The DNN exhibited a similar layered transformation, progressing from cross-correlation to cross-matching principles.

Conclusions:

• Human visual cortex exhibits a hierarchical organization for depth perception, transitioning from cross-correlation to cross-matching computations.
• The study reveals shared computational principles between human and artificial visual systems for robust 3D vision.
• This brain-DNN alignment provides insights into the neural basis of stereo vision.