Zero-shot counting with a dual-stream neural network model.

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Cognitive science

Background:

• Visual scene understanding requires object recognition and relational structure encoding.
• Primate brains utilize dual processing streams (ventral and dorsal) for these functions.
• Current deep learning models excel at object recognition but struggle with scene structure and numerosity encoding.

Purpose of the Study:

• To develop a deep learning model that can encode visual scene structure, including numerosity, in a manner consistent with primate brain function.
• To achieve zero-shot counting capabilities for unfamiliar objects within complex scenes.

Main Methods:

• A dual-stream deep learning network architecture was designed, inspired by the primate brain's ventral and dorsal streams.
• The model was trained to process visual scenes and predict object counts.
• Model performance was evaluated on its ability to count unfamiliar objects (zero-shot learning).

Main Results:

• The dual-stream network successfully performed zero-shot counting of objects in visual scenes.
• The model generated spatial response fields and lognormal number codes similar to those found in the macaque posterior parietal cortex.
• The model accurately predicted human counting behavior.

Conclusions:

• The developed dual-stream network provides a computational model for understanding how the primate brain encodes visual scene structure and numerosity.
• The findings support an enactive theory of the posterior parietal cortex's role in visual scene comprehension.
• This research bridges artificial intelligence and neuroscience to explain complex visual processing.