An image-computable model of speeded decision-making.

Area of Science:

• Cognitive Neuroscience
• Computational Vision
• Decision Making

Background:

• Evidence accumulation models (EAMs) excel at modeling response time (RT) data but lack mechanisms for visual representation extraction.
• Existing models do not fully explain the link between raw visual input and abstract perceptual representations used in decision-making.

Purpose of the Study:

• To bridge the gap between visual processing and decision-making models by integrating convolutional neural networks (CNNs) with EAMs.
• To develop a unified framework, the visual accumulator model (VAM), for analyzing trial-level RTs and raw visual stimuli.
• To investigate how the visual system extracts task-relevant representations for guiding decisions.

Main Methods:

• Jointly fitted CNNs for visual processing and EAMs within a unified Bayesian framework.
• Utilized large-scale cognitive training data from a stylized flanker task, including individual subject RTs and pixel-space visual stimuli.
• Employed a probabilistic framework to constrain neural network models with behavioral data.

Main Results:

• The visual accumulator model (VAM) successfully captured individual differences in congruency effects, RTs, and accuracy.
• Demonstrated that task-relevant information selection involves the orthogonalization of relevant and irrelevant visual representations.
• Showcased the framework's ability to relate neural network-derived visual representations to observable behavioral outputs.

Conclusions:

• The VAM provides a novel probabilistic framework for understanding visual representation extraction in decision-making tasks.
• This approach enables the joint analysis of neural network models of vision and behavioral data.
• The study elucidates the mechanisms by which the visual system generates representations that guide cognitive processes.