Neuronal tuning aligns dynamically with object and texture manifolds across the visual hierarchy.

Area of Science:

• Neuroscience
• Computational Vision
• Artificial Intelligence

Background:

• Neurons in the visual cortex respond to diverse stimuli, but the principles governing these responses in higher visual areas remain unclear.
• While primary visual cortex (V1) neurons tune to simple features, this framework is insufficient for understanding complex feature encoding in areas like V4 and the posterior inferotemporal cortex (PIT).
• Current vision models struggle to fully explain the complex feature representations in the ventral stream.

Purpose of the Study:

• To investigate what visual features neurons prioritize in different areas of the ventral stream.
• To compare texture-based versus object structure-based feature prioritization.
• To understand the emergence of object alignment in neural responses.

Main Methods:

• Utilized generative models (deep networks) to synthesize novel images guided by neural activity.
• Employed closed-loop optimization, allowing neurons in V1, V4, and PIT to direct image synthesis.
• Compared neural responses to images optimized for texture versus object structure.

Main Results:

• Neurons in V1 and V4 showed stronger alignment with texture-based image spaces.
• Many neurons in PIT responded similarly to both texture- and object-optimized images.
• Evidence suggests a shift towards prioritizing shared local motifs rather than whole-object templates in PIT, with object alignment developing later.

Conclusions:

• Neural coding in the ventral stream emphasizes shared local motifs, particularly in higher areas like PIT.
• Object representation emerges later in the ventral stream, challenging template-based models.
• Findings provide insights into visual coding principles and the limitations of current computational vision models.