Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• Recurrent neural networks (RNNs) are crucial for modeling dynamical computations in neuroscience and machine learning.
• Understanding how neural systems solve tasks requires mechanistic insights into trained networks.
• Solution degeneracy, where different networks exhibit distinct internal solutions for the same task, poses a challenge for interpretability.

Purpose of the Study:

• To develop a unified framework for systematically quantifying and controlling solution degeneracy in task-trained RNNs.
• To investigate the impact of task complexity, learning, network size, and regularization on degeneracy.
• To provide tools for uncovering shared neural mechanisms and modeling biological neural variability.

Main Methods:

• Developed a unified framework to quantify and control solution degeneracy across behavior, neural dynamics, and weight space.
• Trained and analyzed 3,400 RNNs on four neuroscience-relevant tasks (flip-flop memory, sine wave generation, delayed discrimination, path integration).
• Systematically varied task complexity, learning regime, network size, and regularization.

Main Results:

• Higher task complexity and feature learning decreased neural dynamics degeneracy but increased weight space degeneracy.
• Larger network size and structural regularization consistently reduced degeneracy across all levels (behavior, dynamics, weight space).
• Findings empirically validate the Contravariance Principle.

Conclusions:

• The developed framework offers a principled approach to quantifying and controlling solution degeneracy in RNNs.
• Results provide practical guidance for tuning RNN solutions to either reveal shared neural mechanisms or model biological variability.
• This work enhances the development of interpretable and biologically grounded models of neural computation.