Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Recurrent neural networks (RNNs) are crucial for modeling dynamical computations in neuroscience and machine learning.
• Understanding how these networks solve tasks is challenging due to 'solution degeneracy,' where different networks exhibit distinct internal mechanisms despite similar performance.

Purpose of the Study:

• To develop a unified framework for systematically quantifying and controlling solution degeneracy in task-trained RNNs.
• To investigate the impact of task complexity, learning, network size, and regularization on solution degeneracy.

Main Methods:

• Trained 3,400 RNNs on four neuroscience-relevant tasks (flip-flop memory, sine wave generation, delayed discrimination, path integration).
• Systematically varied task complexity, learning regime, network size, and regularization.
• Developed a framework to quantify degeneracy across behavioral, neural dynamics, and weight space levels.

Main Results:

• Higher task complexity and feature learning decreased neural dynamics degeneracy but increased weight space degeneracy.
• Larger networks and structural regularization consistently reduced degeneracy across all levels.
• Findings empirically validate the Contravariance Principle.

Conclusions:

• The developed framework provides a principled approach to quantifying and controlling solution degeneracy in RNNs.
• Offers practical guidance for tailoring RNN solutions to uncover shared neural mechanisms or model biological variability.
• Enhances the interpretability and biological grounding of neural computation models.