Recurrent neural networks that learn multi-step visual routines with reinforcement learning

Area of Science:

• Computational neuroscience
• Cognitive science
• Artificial intelligence

Background:

• Cognitive tasks are often solved sequentially, requiring intermediate results to be stored and propagated.
• Working memory in the visual cortex enhances neural activity for storing and accessing intermediate results.

Purpose of the Study:

• To investigate how elemental visual operations and their sequencing can emerge in neural networks trained with reinforcement learning.
• To propose a new recurrent neural network architecture capable of learning composite visual tasks.

Main Methods:

• Developed a novel recurrent neural network architecture.
• Trained the network using RELEARNN, a biologically plausible four-factor Hebbian learning rule.
• Utilized three visual tasks with available electrophysiological recordings from monkey visual cortex.

Main Results:

• Networks learned elemental operations like contour grouping and visual search.
• The network successfully executed sequences of operations based on visual stimuli and task rewards.
• Trained network activity showed enhanced responses to behaviorally relevant stimuli, mirroring monkey visual cortex activity.
• Relevant information was maintained as enhanced activity and passed between subroutines.

Conclusions:

• Biologically plausible learning rules can train recurrent neural networks for multistep visual tasks.
• The proposed architecture and learning rule offer a model for understanding sequential processing in neural systems.
• This work bridges the gap between reinforcement learning and biologically inspired neural computation for complex cognitive functions.