Reinforcement learning in densely recurrent biological networks

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Evolutionary computation

Background:

• Training recurrent neural networks (RNNs) is challenging due to gradient issues (exploding/vanishing) and slow convergence of evolutionary methods.
• Existing methods struggle with efficient optimization of complex, biologically inspired neural architectures.

Purpose of the Study:

• To introduce a novel, gradient-free optimization framework for training recurrent neural networks.
• To benchmark the proposed method on biologically relevant tasks using the Caenorhabditis elegans connectome.
• To investigate the efficacy of biologically derived weight priors in refining neural networks.

Main Methods:

• Developed Evolutionary Nonlinear Optimization with Mesh Adaptive Direct search (ENOMAD), a hybrid framework combining evolutionary exploration and direct search exploitation.
• Implemented reinforcement learning principles within the gradient-free optimization process.
• Utilized the neural connectome of Caenorhabditis elegans and food-foraging tasks for benchmarking.

Main Results:

• ENOMAD successfully trained recurrent networks, significantly outperforming the untrained native circuitry.
• The method demonstrated efficient specialization of natural recurrent networks for specific tasks.
• Biologically derived weight priors enabled refinement rather than complete rebuilding of the neural circuitry, showcasing transfer learning.

Conclusions:

• Integrating evolutionary search with nonlinear optimization offers an efficient, biologically grounded strategy for RNN specialization.
• ENOMAD provides a powerful tool for advancing research in computational neuroscience and AI.
• The framework facilitates the study of learning and adaptation in biological and artificial neural systems.