Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Area of Science:

• Computational neuroscience
• Machine learning
• Dynamical systems

Background:

• Recurrent neural networks (RNNs) are prevalent in computational neuroscience and machine learning.
• Standard RNN models may oversimplify real neuronal network computations.

Purpose of the Study:

• To enhance RNN computational power by integrating Bayesian inference techniques.
• To develop a novel RNN model for more effective processing of dynamic inputs.

Main Methods:

• Utilized an RNN as a generative model for environmental dynamic inputs (e.g., speech, kinematics).
• Derived Bayesian update equations to decode RNN outputs, defining a 'recognizing RNN' (rRNN).
• Implemented a predictive coding scheme within the rRNN for dynamic inputs.

Main Results:

• The rRNN demonstrates enhanced computational capabilities compared to conventional RNNs.
• rRNNs exhibit faster decoding of dynamic stimuli and robustness to initial conditions and noise.
• Successfully applied rRNN for online decoding of human kinematics.

Conclusions:

• The Bayesian inversion of RNNs offers a computationally powerful approach for modeling brain function.
• The rRNN serves as a valuable machine learning tool for dynamic input recognition.
• This fusion advances predictive coding mechanisms for dynamic sensory information processing.