Finite state automata resulting from temporal information maximization and a temporal learning rule

Area of Science:

• Computational Neuroscience
• Information Theory
• Machine Learning

Background:

• Linkser's Infomax principle applies to feedforward neural networks.
• Recurrent systems possess complex spatial and temporal signal properties.
• Stochastic interdependence requires novel measures for analysis.

Purpose of the Study:

• Extend Infomax to a measure of stochastic interdependence for recurrent systems.
• Analyze Temporal Infomax on constrained Markov chains with external inputs.
• Investigate information flow and learning rules in these systems.

Main Methods:

• Quantify stochastic interaction using Kullback-Leibler divergence.
• Analyze constrained Markov chains with clamped input units.
• Employ computer simulations and analytical confirmations.
• Numerically assess information flow and temporal learning rules.

Main Results:

• Temporal Infomax on constrained chains yields finite state automata (deterministic or weakly nondeterministic).
• Internal state transitions are predictable given full state and input, despite random single-unit activity.
• High information flow observed from input to internal units.
• A simple temporal learning rule approximates Temporal Infomax optimization.

Conclusions:

• Temporal Infomax provides a framework for understanding signal processing in recurrent neural networks.
• The developed measure and learning rule offer insights into biological neural correlations.
• Results suggest predictable system dynamics emerge from locally random components.