Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks

Area of Science:

• Computational neuroscience
• Machine learning
• Dynamical systems theory

Background:

• Recurrent Neural Networks (RNNs) excel at modeling time-varying data but operate as black boxes.
• Understanding the internal mechanisms of trained RNNs is crucial for reliable application.
• Existing methods lack transparency into RNN computation.

Purpose of the Study:

• To investigate the hypothesis that fixed points and linearized dynamics reveal RNN computational mechanisms.
• To develop a method for analyzing the internal dynamics of trained RNNs.
• To demonstrate the utility of linearization in understanding RNN behavior.

Main Methods:

• Developed an optimization technique to identify stable/unstable fixed points and slow manifolds in RNN phase space.
• Applied linearization around these identified points to analyze local dynamics.
• Tested the technique on diverse high-dimensional RNN models, including a flip-flop, sine wave generator, and moving average.

Main Results:

• Fixed points and linearized dynamics around them provide insights into RNN computations.
• The method successfully identified computational mechanisms in tested RNN examples.
• Linearization in slow regions of phase space is effective for analysis.

Conclusions:

• Fixed and slow points, along with linearized dynamics, offer a powerful tool for reverse-engineering RNNs.
• This approach demystifies RNNs, enabling better understanding and trust.
• The proposed technique is broadly applicable to various high-dimensional RNN architectures.