Multiplicative couplings facilitate rapid learning and information gating in recurrent neural networks

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• The mammalian forebrain, responsible for higher cognition, exhibits architectural similarities to modern machine learning systems, with the cortex resembling Recurrent Neural Networks (RNNs) and the thalamus resembling Feedforward Neural Networks (FNNs).
• The precise mechanisms by which these architectural features contribute to the forebrain's learning capacity remain largely unknown.

Purpose of the Study:

• To investigate the functional role of thalamocortical interactions in learning and computation.
• To develop a novel computational mechanism inspired by thalamocortical architecture to enhance learning in artificial neural networks.

Main Methods:

• Development of a multiplicative coupling mechanism between RNN and FNN architectures.
• Implementation of Hebbian-weight amplification and synaptic-neuronal coupling for context-dependent gating and rapid switching.
• Demonstration of multiplicative feedback-driven synaptic plasticity in supervised, reinforcement, and unsupervised learning paradigms.

Main Results:

• Achieved 2-100 fold speed improvements in various learning settings, enhancing memory capacity, model robustness, and generalization in RNNs.
• Validated the biological plausibility and efficacy of multiplicative gating in modeling complex brain circuits, including decision-making, working memory, and navigation.
• Demonstrated multi-plastic attractor dynamics and enhanced computation in recurrent neural circuits through neuroscience-inspired approaches.

Conclusions:

• The proposed multiplicative coupling mechanism offers significant computational advantages for artificial intelligence, drawing parallels to biological neural processing.
• This research provides profound insights into neuroscience-inspired computation, highlighting the potential of mimicking brain structures for advanced AI capabilities.