Structured flexibility in recurrent neural networks via neuromodulation.

Area of Science:

• Theoretical Neuroscience
• Computational Neuroscience
• Artificial Intelligence

Background:

• Understanding biological intelligence requires sophisticated computational models.
• Recurrent neural networks (RNNs) model brain computations but often assume fixed synaptic weights.
• Neuromodulators dynamically alter synaptic weights in biological systems.

Purpose of the Study:

• To explore the computational implications of synaptic gain scaling, a form of neuromodulation.
• To introduce a neuromodulated RNN (NM-RNN) model that incorporates dynamic synaptic scaling.
• To investigate how neuromodulation impacts network training, generalization, and computational mechanisms.

Main Methods:

• Developed a neuromodulated RNN (NM-RNN) model where a subnetwork outputs a signal to scale recurrent weights.
• Utilized task-optimized low-rank RNNs for model implementation.
• Conducted empirical experiments on canonical tasks and performed theoretical analyses.

Main Results:

• The NM-RNN model achieved higher accuracy in training and generalization compared to standard low-rank RNNs.
• Neuromodulatory gain scaling was shown to enable gating mechanisms similar to those in artificial RNNs.
• Analysis revealed how task computations are distributed within the low-rank dynamics of trained NM-RNNs.

Conclusions:

• Synaptic gain scaling provides structured flexibility, enhancing RNN performance.
• Neuromodulation offers a mechanism for implementing flexible gating in neural networks.
• The NM-RNN model provides a framework for understanding neuromodulation's role in biological and artificial intelligence.