Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Accurate prediction is essential for cognitive functions like decision-making, perception, and language.
• The brain excels at prediction in dynamic and uncertain environments, but the underlying computational mechanisms remain a challenge.
• Bayesian inference offers optimal predictions but is computationally intractable for complex systems.

Purpose of the Study:

• To identify a computational architecture enabling accurate and efficient predictions in stochastic, changing, and structured environments.
• To explore how recurrent neural networks can replicate the brain's predictive abilities.
• To investigate the role of specific neural mechanisms in enabling robust predictive processing.

Main Methods:

• Developed and analyzed a specific recurrent neural network (RNN) architecture.
• Incorporated three key mechanisms: gating, lateral connections, and recurrent weight training.
• Evaluated network performance across several simulated environments.

Main Results:

• The proposed RNN architecture demonstrated simple and accurate predictive solutions.
• Networks developed internal representations of environmental latent variables and their precision.
• The architecture adapted learning rates to environmental changes without altering connection weights, mirroring optimal solutions and brain function.
• Leveraged multiple levels of latent structure for enhanced prediction.

Conclusions:

• Gated recurrence in RNNs provides a viable computational framework for prediction in complex, real-world environments.
• This architecture offers a potential explanation for the brain's sophisticated predictive capabilities.
• Gated recurrence may serve as a fundamental building block for predictive processing in biological and artificial systems.